S3: Implement IAM defaults and STS signing key fallback (#8348)

* S3: Implement IAM defaults and STS signing key fallback logic

* S3: Refactor startup order to init SSE-S3 key manager before IAM

* S3: Derive STS signing key from KEK using HKDF for security isolation

* S3: Document STS signing key fallback in security.toml

* fix(s3api): refine anonymous access logic and secure-by-default behavior

- Initialize anonymous identity by default in `NewIdentityAccessManagement` to prevent nil pointer exceptions.
- Ensure `ReplaceS3ApiConfiguration` preserves the anonymous identity if not present in the new configuration.
- Update `NewIdentityAccessManagement` signature to accept `filerClient`.
- In legacy mode (no policy engine), anonymous defaults to Deny (no actions), preserving secure-by-default behavior.
- Use specific `LookupAnonymous` method instead of generic map lookup.
- Update tests to accommodate signature changes and verify improved anonymous handling.

* feat(s3api): make IAM configuration optional

- Start S3 API server without a configuration file if `EnableIam` option is set.
- Default to `Allow` effect for policy engine when no configuration is provided (Zero-Config mode).
- Handle empty configuration path gracefully in `loadIAMManagerFromConfig`.
- Add integration test `iam_optional_test.go` to verify empty config behavior.

* fix(iamapi): fix signature mismatch in NewIdentityAccessManagementWithStore

* fix(iamapi): properly initialize FilerClient instead of passing nil

* fix(iamapi): properly initialize filer client for IAM management

- Instead of passing `nil`, construct a `wdclient.FilerClient` using the provided `Filers` addresses.
- Ensure `NewIdentityAccessManagementWithStore` receives a valid `filerClient` to avoid potential nil pointer dereferences or limited functionality.

* clean: remove dead code in s3api_server.go

* refactor(s3api): improve IAM initialization, safety and anonymous access security

* fix(s3api): ensure IAM config loads from filer after client init

* fix(s3): resolve test failures in integration, CORS, and tagging tests

- Fix CORS tests by providing explicit anonymous permissions config
- Fix S3 integration tests by setting admin credentials in init
- Align tagging test credentials in CI with IAM defaults
- Added goroutine to retry IAM config load in iamapi server

* fix(s3): allow anonymous access to health targets and S3 Tables when identities are present

* fix(ci): use /healthz for Caddy health check in awscli tests

* iam, s3api: expose DefaultAllow from IAM and Policy Engine

This allows checking the global "Open by Default" configuration from
other components like S3 Tables.

* s3api/s3tables: support DefaultAllow in permission logic and handler

Updated CheckPermissionWithContext to respect the DefaultAllow flag
in PolicyContext. This enables "Open by Default" behavior for
unauthenticated access in zero-config environments. Added a targeted
unit test to verify the logic.

* s3api/s3tables: propagate DefaultAllow through handlers

Propagated the DefaultAllow flag to individual handlers for
namespaces, buckets, tables, policies, and tagging. This ensures
consistent "Open by Default" behavior across all S3 Tables API
endpoints.

* s3api: wire up DefaultAllow for S3 Tables API initialization

Updated registerS3TablesRoutes to query the global IAM configuration
and set the DefaultAllow flag on the S3 Tables API server. This
completes the end-to-end propagation required for anonymous access in
zero-config environments. Added a SetDefaultAllow method to
S3TablesApiServer to facilitate this.

* s3api: fix tests by adding DefaultAllow to mock IAM integrations

The IAMIntegration interface was updated to include DefaultAllow(),
breaking several mock implementations in tests. This commit fixes
the build errors by adding the missing method to the mocks.

* env

* ensure ports

* env

* env

* fix default allow

* add one more test using non-anonymous user

* debug

* add more debug

* less logs
This commit is contained in:
Chris Lu
2026-02-16 13:59:13 -08:00
committed by GitHub
parent cc58272219
commit 0d8588e3ae
46 changed files with 1084 additions and 109 deletions

View File

@@ -111,7 +111,7 @@ jobs:
# Wait for S3 API # Wait for S3 API
for i in {1..30}; do for i in {1..30}; do
if curl -s http://localhost:8333/ > /dev/null 2>&1; then if curl -s http://localhost:8333/healthz > /dev/null 2>&1; then
echo "✓ S3 API is ready" echo "✓ S3 API is ready"
break break
fi fi

View File

@@ -461,6 +461,8 @@ jobs:
export S3_ENDPOINT="http://localhost:8006" export S3_ENDPOINT="http://localhost:8006"
export S3_ACCESS_KEY="0555b35654ad1656d804" export S3_ACCESS_KEY="0555b35654ad1656d804"
export S3_SECRET_KEY="h7GhxuBLTrlhVUyxSPUKUV8r/2EI4ngqJxD7iBdBYLhwluN30JaT3Q==" export S3_SECRET_KEY="h7GhxuBLTrlhVUyxSPUKUV8r/2EI4ngqJxD7iBdBYLhwluN30JaT3Q=="
export AWS_ACCESS_KEY_ID="$S3_ACCESS_KEY"
export AWS_SECRET_ACCESS_KEY="$S3_SECRET_KEY"
# Run the specific test that is equivalent to AWS S3 tagging behavior # Run the specific test that is equivalent to AWS S3 tagging behavior
make test-with-server || { make test-with-server || {

View File

@@ -97,7 +97,7 @@ jobs:
# Verify service accessibility # Verify service accessibility
echo "=== Verifying Service Accessibility ===" echo "=== Verifying Service Accessibility ==="
curl -f http://localhost:8080/realms/master curl -f http://localhost:8080/realms/master
curl -s http://localhost:8333 curl -s http://localhost:8333/healthz
echo "✅ SeaweedFS S3 API is responding (IAM-protected endpoint)" echo "✅ SeaweedFS S3 API is responding (IAM-protected endpoint)"
# Run Keycloak-specific tests # Run Keycloak-specific tests

View File

@@ -35,7 +35,7 @@ jobs:
set -e set -e
mkdir -p /tmp/data mkdir -p /tmp/data
./weed -v=3 server -s3 -dir=/tmp/data -s3.config=../docker/compose/s3.json -master.peers=none > weed.log 2>&1 & ./weed -v=3 server -s3 -dir=/tmp/data -s3.config=../docker/compose/s3.json -master.peers=none > weed.log 2>&1 &
until curl -s http://localhost:8333/ > /dev/null; do sleep 1; done until curl -s http://localhost:8333/healthz > /dev/null; do sleep 1; done
- name: Setup Caddy - name: Setup Caddy
run: | run: |
@@ -54,7 +54,7 @@ jobs:
- name: Start Caddy - name: Start Caddy
run: | run: |
./caddy start ./caddy start
until curl -fsS --insecure https://localhost:8443 > /dev/null; do sleep 1; done until curl -fsS --insecure https://localhost:8443/healthz > /dev/null; do sleep 1; done
- name: Create Bucket - name: Create Bucket
run: | run: |

View File

@@ -12,6 +12,7 @@ FILER_PORT := 8888
TEST_TIMEOUT := 10m TEST_TIMEOUT := 10m
TEST_PATTERN := TestCORS TEST_PATTERN := TestCORS
SERVER_DIR := test-mini-data SERVER_DIR := test-mini-data
S3_CONFIG := s3_test_config.json
# Default target # Default target
help: help:
@@ -80,13 +81,15 @@ start-server: check-deps
@echo "🔍 DEBUG: Creating volume directory..." @echo "🔍 DEBUG: Creating volume directory..."
@mkdir -p $(SERVER_DIR) @mkdir -p $(SERVER_DIR)
@echo "🔍 DEBUG: Launching SeaweedFS S3 server in background..." @echo "🔍 DEBUG: Launching SeaweedFS S3 server in background..."
@echo "🔍 DEBUG: Command: $(WEED_BINARY) mini -dir=$(SERVER_DIR) -s3.port=$(S3_PORT) -s3.config=$(S3_CONFIG)" @echo "🔍 DEBUG: Command: AWS_ACCESS_KEY_ID=some_access_key1 AWS_SECRET_ACCESS_KEY=some_secret_key1 $(WEED_BINARY) mini -dir=$(SERVER_DIR) -s3.port=$(S3_PORT) -s3.config=$(S3_CONFIG)"
@$(WEED_BINARY) mini \ @env AWS_ACCESS_KEY_ID=some_access_key1 \
AWS_SECRET_ACCESS_KEY=some_secret_key1 \
$(WEED_BINARY) mini \
-dir=$(SERVER_DIR) \ -dir=$(SERVER_DIR) \
-s3.port=$(S3_PORT) \ -s3.port=$(S3_PORT) \
-s3.config=$(S3_CONFIG) \ -s3.config=$(S3_CONFIG) \
> weed-test.log 2>&1 & \ > weed-test.log 2>&1 & \
echo $$! > weed-test.pid echo $$! > weed-server.pid
@echo "Waiting for S3 server to be ready..." @echo "Waiting for S3 server to be ready..."
@for i in $$(seq 1 30); do \ @for i in $$(seq 1 30); do \
@@ -97,7 +100,7 @@ start-server: check-deps
sleep 1; \ sleep 1; \
done; \ done; \
echo "S3 server failed to start"; \ echo "S3 server failed to start"; \
exit 1 > weed-server.pid exit 1
@echo "🔍 DEBUG: Server PID: $$(cat weed-server.pid 2>/dev/null || echo 'PID file not found')" @echo "🔍 DEBUG: Server PID: $$(cat weed-server.pid 2>/dev/null || echo 'PID file not found')"
@echo "🔍 DEBUG: Checking if PID is still running..." @echo "🔍 DEBUG: Checking if PID is still running..."
@sleep 2 @sleep 2

View File

@@ -0,0 +1,27 @@
{
"identities": [
{
"name": "anonymous",
"actions": [
"Read",
"List"
]
},
{
"name": "admin",
"credentials": [
{
"accessKey": "some_access_key1",
"secretKey": "some_secret_key1"
}
],
"actions": [
"Admin",
"Read",
"List",
"Tagging",
"Write"
]
}
]
}

View File

@@ -1,7 +1,6 @@
package example package example
import ( import (
"os"
"testing" "testing"
"time" "time"
@@ -22,10 +21,8 @@ func TestIAMOperations(t *testing.T) {
// Set credentials before starting cluster // Set credentials before starting cluster
accessKey := "testkey123" accessKey := "testkey123"
secretKey := "testsecret456" secretKey := "testsecret456"
os.Setenv("AWS_ACCESS_KEY_ID", accessKey) t.Setenv("AWS_ACCESS_KEY_ID", accessKey)
os.Setenv("AWS_SECRET_ACCESS_KEY", secretKey) t.Setenv("AWS_SECRET_ACCESS_KEY", secretKey)
defer os.Unsetenv("AWS_ACCESS_KEY_ID")
defer os.Unsetenv("AWS_SECRET_ACCESS_KEY")
// Create and start test cluster // Create and start test cluster
cluster, err := startMiniCluster(t) cluster, err := startMiniCluster(t)

View File

@@ -146,6 +146,14 @@ func startMiniCluster(t *testing.T) (*TestCluster, error) {
return nil, fmt.Errorf("failed to create security.toml: %v", err) return nil, fmt.Errorf("failed to create security.toml: %v", err)
} }
// Set environment variables for admin credentials safely for this test
if os.Getenv("AWS_ACCESS_KEY_ID") == "" {
t.Setenv("AWS_ACCESS_KEY_ID", "admin")
}
if os.Getenv("AWS_SECRET_ACCESS_KEY") == "" {
t.Setenv("AWS_SECRET_ACCESS_KEY", "admin")
}
// Start weed mini in a goroutine by calling the command directly // Start weed mini in a goroutine by calling the command directly
cluster.wg.Add(1) cluster.wg.Add(1)
go func() { go func() {

View File

@@ -216,6 +216,14 @@ enabled = true
err = os.WriteFile(credentialToml, []byte(credentialConfig), 0644) err = os.WriteFile(credentialToml, []byte(credentialConfig), 0644)
require.NoError(t, err) require.NoError(t, err)
// Set environment variables for admin credentials safely for this test
if os.Getenv("AWS_ACCESS_KEY_ID") == "" {
t.Setenv("AWS_ACCESS_KEY_ID", "admin")
}
if os.Getenv("AWS_SECRET_ACCESS_KEY") == "" {
t.Setenv("AWS_SECRET_ACCESS_KEY", "admin")
}
cluster.wg.Add(1) cluster.wg.Add(1)
go func() { go func() {
defer cluster.wg.Done() defer cluster.wg.Done()

View File

@@ -511,8 +511,10 @@ func createTableBucket(t *testing.T, env *TestEnvironment, bucketName string) {
} }
defer resp.Body.Close() defer resp.Body.Close()
body, _ := io.ReadAll(resp.Body)
t.Logf("Create table bucket %s response: status=%d, body=%s", bucketName, resp.StatusCode, string(body))
if resp.StatusCode != http.StatusOK && resp.StatusCode != http.StatusConflict { if resp.StatusCode != http.StatusOK && resp.StatusCode != http.StatusConflict {
body, _ := io.ReadAll(resp.Body)
t.Fatalf("Failed to create table bucket %s, status %d: %s", bucketName, resp.StatusCode, body) t.Fatalf("Failed to create table bucket %s, status %d: %s", bucketName, resp.StatusCode, body)
} }
t.Logf("Created table bucket %s", bucketName) t.Logf("Created table bucket %s", bucketName)

View File

@@ -56,8 +56,6 @@ func TestPyIcebergRestCatalog(t *testing.T) {
cmd := exec.Command("docker", "run", "--rm", cmd := exec.Command("docker", "run", "--rm",
"--add-host", "host.docker.internal:host-gateway", "--add-host", "host.docker.internal:host-gateway",
"-e", fmt.Sprintf("AWS_ACCESS_KEY_ID=%s", "test"),
"-e", fmt.Sprintf("AWS_SECRET_ACCESS_KEY=%s", "test"),
"-e", fmt.Sprintf("AWS_ENDPOINT_URL=%s", s3Endpoint), "-e", fmt.Sprintf("AWS_ENDPOINT_URL=%s", s3Endpoint),
"-v", fmt.Sprintf("%s:/app:ro", testDir), "-v", fmt.Sprintf("%s:/app:ro", testDir),
"iceberg-rest-test", "iceberg-rest-test",
@@ -78,3 +76,69 @@ func TestPyIcebergRestCatalog(t *testing.T) {
t.Errorf("PyIceberg test failed: %v", err) t.Errorf("PyIceberg test failed: %v", err)
} }
} }
// TestPyIcebergRestCatalogAuthenticated tests the Iceberg REST Catalog using PyIceberg with authentication.
// This test uses the default admin credentials that SeaweedFS creates on startup.
func TestPyIcebergRestCatalogAuthenticated(t *testing.T) {
if testing.Short() {
t.Skip("Skipping integration test in short mode")
}
env := NewTestEnvironment(t)
defer env.Cleanup(t)
if !env.dockerAvailable {
t.Skip("Docker not available, skipping PyIceberg integration test")
}
// Use default admin credentials
testAccessKey := "admin"
testSecretKey := "admin"
// Start SeaweedFS (it will use default admin credentials from environment if set)
env.StartSeaweedFS(t)
// Create the test bucket first (using unauthenticated request, which works with DefaultAllow)
bucketName := "pyiceberg-auth-test"
createTableBucket(t, env, bucketName)
// Build the test working directory path
testDir := filepath.Join(env.seaweedDir, "test", "s3tables", "catalog")
// Run PyIceberg test using Docker with authentication
catalogURL := fmt.Sprintf("http://host.docker.internal:%d", env.icebergPort)
s3Endpoint := fmt.Sprintf("http://host.docker.internal:%d", env.s3Port)
warehouse := fmt.Sprintf("s3://%s/", bucketName)
// Build the test image first for faster repeated runs
buildCmd := exec.Command("docker", "build", "-t", "iceberg-rest-test", "-f", "Dockerfile.pyiceberg", ".")
buildCmd.Dir = testDir
if out, err := buildCmd.CombinedOutput(); err != nil {
t.Fatalf("Failed to build test image: %v\n%s", err, string(out))
}
cmd := exec.Command("docker", "run", "--rm",
"--add-host", "host.docker.internal:host-gateway",
"-e", fmt.Sprintf("AWS_ENDPOINT_URL=%s", s3Endpoint),
"-v", fmt.Sprintf("%s:/app:ro", testDir),
"iceberg-rest-test",
"python3", "/app/test_rest_catalog_auth.py",
"--catalog-url", catalogURL,
"--warehouse", warehouse,
"--prefix", bucketName,
"--access-key", testAccessKey,
"--secret-key", testSecretKey,
)
cmd.Dir = testDir
cmd.Stdout = os.Stdout
cmd.Stderr = os.Stderr
t.Logf("Running PyIceberg REST catalog test with authentication...")
t.Logf(" Catalog URL: %s", catalogURL)
t.Logf(" Warehouse: %s", warehouse)
t.Logf(" Access Key: %s", testAccessKey)
if err := cmd.Run(); err != nil {
t.Errorf("PyIceberg authenticated test failed: %v", err)
}
}

View File

@@ -0,0 +1,36 @@
package catalog
import (
"fmt"
"io"
"net/http"
"testing"
)
// verifyTableBucketMetadata verifies that a table bucket was created with proper metadata
func verifyTableBucketMetadata(t *testing.T, env *TestEnvironment, bucketName string) {
t.Helper()
// Use S3Tables REST API to get the bucket
endpoint := fmt.Sprintf("http://localhost:%d/buckets/%s", env.s3Port, bucketName)
req, err := http.NewRequest(http.MethodGet, endpoint, nil)
if err != nil {
t.Fatalf("Failed to create request: %v", err)
}
req.Header.Set("Content-Type", "application/x-amz-json-1.1")
resp, err := http.DefaultClient.Do(req)
if err != nil {
t.Fatalf("Failed to get table bucket %s: %v", bucketName, err)
}
defer resp.Body.Close()
body, _ := io.ReadAll(resp.Body)
t.Logf("Get table bucket %s response: status=%d, body=%s", bucketName, resp.StatusCode, string(body))
if resp.StatusCode != http.StatusOK {
t.Fatalf("Failed to get table bucket %s, status %d: %s", bucketName, resp.StatusCode, body)
}
t.Logf("Verified table bucket %s exists with metadata", bucketName)
}

View File

@@ -201,6 +201,7 @@ def main():
"uri": args.catalog_url, "uri": args.catalog_url,
"warehouse": args.warehouse, "warehouse": args.warehouse,
"prefix": args.prefix, "prefix": args.prefix,
"s3.anonymous": "true", # Disable AWS request signing for unauthenticated access
} }
) )
print(f"Successfully connected to catalog on attempt {attempt + 1}") print(f"Successfully connected to catalog on attempt {attempt + 1}")

View File

@@ -0,0 +1,223 @@
#!/usr/bin/env python3
"""
Iceberg REST Catalog Compatibility Test for SeaweedFS (Authenticated)
This script tests the Iceberg REST Catalog API compatibility with authentication.
Usage:
python3 test_rest_catalog_auth.py --catalog-url http://localhost:8182 \\
--access-key admin --secret-key admin
Requirements:
pip install pyiceberg[s3fs]
"""
import argparse
import sys
from pyiceberg.catalog import load_catalog
from pyiceberg.schema import Schema
from pyiceberg.types import (
IntegerType,
LongType,
StringType,
NestedField,
)
from pyiceberg.exceptions import (
NamespaceAlreadyExistsError,
NoSuchNamespaceError,
TableAlreadyExistsError,
NoSuchTableError,
)
def test_config_endpoint(catalog):
"""Test that the catalog config endpoint returns valid configuration."""
print("Testing /v1/config endpoint...")
# The catalog is already loaded which means config endpoint worked
print(" /v1/config endpoint working")
return True
def test_namespace_operations(catalog, prefix):
"""Test namespace CRUD operations."""
print("Testing namespace operations...")
namespace = (f"{prefix.replace('-', '_')}_auth_test_ns",)
# List initial namespaces
namespaces = catalog.list_namespaces()
print(f" Initial namespaces: {namespaces}")
# Create namespace
try:
catalog.create_namespace(namespace)
print(f" Created namespace: {namespace}")
except NamespaceAlreadyExistsError:
print(f" ! Namespace already exists: {namespace}")
# List namespaces (should include our new one)
namespaces = catalog.list_namespaces()
if namespace in namespaces:
print(" Namespace appears in list")
else:
print(f" Namespace not found in list: {namespaces}")
return False
# Get namespace properties
try:
props = catalog.load_namespace_properties(namespace)
print(f" Loaded namespace properties: {props}")
except NoSuchNamespaceError:
print(f" Failed to load namespace properties")
return False
return True
def test_table_operations(catalog, prefix):
"""Test table CRUD operations."""
print("Testing table operations...")
namespace = (f"{prefix.replace('-', '_')}_auth_test_ns",)
table_name = "auth_test_table"
table_id = namespace + (table_name,)
# Define a simple schema
schema = Schema(
NestedField(field_id=1, name="id", field_type=LongType(), required=True),
NestedField(field_id=2, name="name", field_type=StringType(), required=False),
NestedField(field_id=3, name="age", field_type=IntegerType(), required=False),
)
# Create table
try:
table = catalog.create_table(
identifier=table_id,
schema=schema,
)
print(f" Created table: {table_id}")
except TableAlreadyExistsError:
print(f" ! Table already exists: {table_id}")
_ = catalog.load_table(table_id)
# List tables
tables = catalog.list_tables(namespace)
if table_name in [t[1] for t in tables]:
print(" Table appears in list")
else:
print(f" Table not found in list: {tables}")
return False
# Load table
try:
loaded_table = catalog.load_table(table_id)
print(f" Loaded table: {loaded_table.name()}")
print(f" Schema: {loaded_table.schema()}")
print(f" Location: {loaded_table.location()}")
except NoSuchTableError:
print(f" Failed to load table")
return False
return True
def test_cleanup(catalog, prefix):
"""Test table and namespace deletion."""
print("Testing cleanup operations...")
namespace = (f"{prefix.replace('-', '_')}_auth_test_ns",)
table_id = namespace + ("auth_test_table",)
# Drop table
try:
catalog.drop_table(table_id)
print(f" Dropped table: {table_id}")
except NoSuchTableError:
print(f" ! Table already deleted: {table_id}")
# Drop namespace
try:
catalog.drop_namespace(namespace)
print(f" Dropped namespace: {namespace}")
except NoSuchNamespaceError:
print(f" ! Namespace already deleted: {namespace}")
except Exception as e:
print(f" ? Namespace drop error (may be expected): {e}")
return True
def main():
parser = argparse.ArgumentParser(description="Test Iceberg REST Catalog with authentication")
parser.add_argument("--catalog-url", required=True, help="Iceberg REST Catalog URL")
parser.add_argument("--warehouse", default="s3://iceberg-test/", help="Warehouse location")
parser.add_argument("--prefix", required=True, help="Table bucket prefix")
parser.add_argument("--access-key", required=True, help="AWS Access Key ID")
parser.add_argument("--secret-key", required=True, help="AWS Secret Access Key")
parser.add_argument("--skip-cleanup", action="store_true", help="Skip cleanup at the end")
args = parser.parse_args()
print(f"Connecting to Iceberg REST Catalog at: {args.catalog_url}")
print(f"Warehouse: {args.warehouse}")
print(f"Prefix: {args.prefix}")
print(f"Using authenticated access with key: {args.access_key}")
print()
# Load the REST catalog with authentication
import time
max_retries = 10
catalog = None
for attempt in range(max_retries):
try:
catalog = load_catalog(
"rest",
**{
"type": "rest",
"uri": args.catalog_url,
"warehouse": args.warehouse,
"prefix": args.prefix,
"s3.access-key-id": args.access_key,
"s3.secret-access-key": args.secret_key,
}
)
print(f"Successfully connected to catalog on attempt {attempt + 1}")
break
except Exception as e:
if attempt < max_retries - 1:
print(f" Attempt {attempt + 1} failed, retrying in 2s... ({e})")
time.sleep(2)
else:
print(f" All {max_retries} attempts failed.")
raise e
# Run tests
tests = [
("Config Endpoint", lambda: test_config_endpoint(catalog)),
("Namespace Operations", lambda: test_namespace_operations(catalog, args.prefix)),
("Table Operations", lambda: test_table_operations(catalog, args.prefix)),
]
if not args.skip_cleanup:
tests.append(("Cleanup", lambda: test_cleanup(catalog, args.prefix)))
passed = 0
failed = 0
for name, test_fn in tests:
print(f"\n{'='*50}")
try:
if test_fn():
passed += 1
print(f"PASSED: {name}")
else:
failed += 1
print(f"FAILED: {name}")
except Exception as e:
failed += 1
print(f"ERROR in {name}: {e}")
print(f"\n{'='*50}")
print(f"Results: {passed} passed, {failed} failed")
return 0 if failed == 0 else 1
if __name__ == "__main__":
sys.exit(main())

View File

@@ -556,6 +556,14 @@ func startMiniCluster(t *testing.T) (*TestCluster, error) {
return nil, fmt.Errorf("failed to create security.toml: %v", err) return nil, fmt.Errorf("failed to create security.toml: %v", err)
} }
// Set environment variables for admin credentials safely for this test
if os.Getenv("AWS_ACCESS_KEY_ID") == "" {
t.Setenv("AWS_ACCESS_KEY_ID", "admin")
}
if os.Getenv("AWS_SECRET_ACCESS_KEY") == "" {
t.Setenv("AWS_SECRET_ACCESS_KEY", "admin")
}
// Start weed mini in a goroutine by calling the command directly // Start weed mini in a goroutine by calling the command directly
cluster.wg.Add(1) cluster.wg.Add(1)
go func() { go func() {

View File

@@ -355,6 +355,9 @@ func isFlagPassed(name string) bool {
// isPortOpenOnIP checks if a port is available for binding on a specific IP address // isPortOpenOnIP checks if a port is available for binding on a specific IP address
func isPortOpenOnIP(ip string, port int) bool { func isPortOpenOnIP(ip string, port int) bool {
if port <= 0 || port > 65535 {
return false
}
listener, err := net.Listen("tcp", fmt.Sprintf("%s:%d", ip, port)) listener, err := net.Listen("tcp", fmt.Sprintf("%s:%d", ip, port))
if err != nil { if err != nil {
return false return false
@@ -366,6 +369,9 @@ func isPortOpenOnIP(ip string, port int) bool {
// isPortAvailable checks if a port is available on any interface // isPortAvailable checks if a port is available on any interface
// This is more comprehensive than checking a single IP // This is more comprehensive than checking a single IP
func isPortAvailable(port int) bool { func isPortAvailable(port int) bool {
if port <= 0 || port > 65535 {
return false
}
// Try to listen on all interfaces (0.0.0.0) // Try to listen on all interfaces (0.0.0.0)
listener, err := net.Listen("tcp", fmt.Sprintf(":%d", port)) listener, err := net.Listen("tcp", fmt.Sprintf(":%d", port))
if err != nil { if err != nil {
@@ -381,6 +387,10 @@ func isPortAvailable(port int) bool {
func findAvailablePortOnIP(ip string, startPort int, maxAttempts int, reservedPorts map[int]bool) int { func findAvailablePortOnIP(ip string, startPort int, maxAttempts int, reservedPorts map[int]bool) int {
for i := 0; i < maxAttempts; i++ { for i := 0; i < maxAttempts; i++ {
port := startPort + i port := startPort + i
if port > 65535 {
// Wrap around to a lower range if we exceed 65535
port = 10000 + (port % 65535)
}
// Skip ports reserved for gRPC calculation // Skip ports reserved for gRPC calculation
if reservedPorts[port] { if reservedPorts[port] {
continue continue
@@ -398,15 +408,15 @@ func findAvailablePortOnIP(ip string, startPort int, maxAttempts int, reservedPo
// If the port is not available, it finds the next available port and updates the pointer // If the port is not available, it finds the next available port and updates the pointer
// The reservedPorts map contains ports that should not be allocated (for gRPC collision avoidance) // The reservedPorts map contains ports that should not be allocated (for gRPC collision avoidance)
func ensurePortAvailableOnIP(portPtr *int, serviceName string, ip string, reservedPorts map[int]bool, flagName string) error { func ensurePortAvailableOnIP(portPtr *int, serviceName string, ip string, reservedPorts map[int]bool, flagName string) error {
if portPtr == nil { // Check if this port was explicitly specified by the user (from CLI, before config file was applied)
isExplicitPort := explicitPortFlags[flagName]
if *portPtr == 0 {
return nil return nil
} }
original := *portPtr original := *portPtr
// Check if this port was explicitly specified by the user (from CLI, before config file was applied)
isExplicitPort := explicitPortFlags[flagName]
// Skip if this port is reserved for gRPC calculation // Skip if this port is reserved for gRPC calculation
if reservedPorts[original] { if reservedPorts[original] {
if isExplicitPort { if isExplicitPort {

View File

@@ -43,6 +43,7 @@ expires_after_seconds = 10 # seconds
# If this JWT key is configured, Filer only accepts writes over HTTP if they are signed with this JWT: # If this JWT key is configured, Filer only accepts writes over HTTP if they are signed with this JWT:
# - f.e. the S3 API Shim generates the JWT # - f.e. the S3 API Shim generates the JWT
# - the Filer server validates the JWT on writing # - the Filer server validates the JWT on writing
# NOTE: This key is ALSO used as a fallback signing key for S3 STS if s3.iam.config does not specify a signingKey.
# the jwt defaults to expire after 10 seconds. # the jwt defaults to expire after 10 seconds.
[jwt.filer_signing] [jwt.filer_signing]
key = "" key = ""

View File

@@ -654,6 +654,14 @@ func (m *IAMManager) GetSTSService() *sts.STSService {
return m.stsService return m.stsService
} }
// DefaultAllow returns whether the default effect is Allow
func (m *IAMManager) DefaultAllow() bool {
if !m.initialized || m.policyEngine == nil {
return true // Default to true if not initialized
}
return m.policyEngine.DefaultAllow()
}
// parseJWTTokenForTrustPolicy parses a JWT token to extract claims for trust policy evaluation // parseJWTTokenForTrustPolicy parses a JWT token to extract claims for trust policy evaluation
func parseJWTTokenForTrustPolicy(tokenString string) (map[string]interface{}, error) { func parseJWTTokenForTrustPolicy(tokenString string) (map[string]interface{}, error) {
// Simple JWT parsing without verification (for trust policy context only) // Simple JWT parsing without verification (for trust policy context only)

View File

@@ -324,6 +324,14 @@ func (e *PolicyEngine) IsInitialized() bool {
return e.initialized return e.initialized
} }
// DefaultAllow returns whether the default effect is Allow
func (e *PolicyEngine) DefaultAllow() bool {
if e.config == nil {
return true // Default to Allow if not configured
}
return e.config.DefaultEffect == string(EffectAllow)
}
// AddPolicy adds a policy to the engine (filerAddress ignored for memory stores) // AddPolicy adds a policy to the engine (filerAddress ignored for memory stores)
func (e *PolicyEngine) AddPolicy(filerAddress string, name string, policy *PolicyDocument) error { func (e *PolicyEngine) AddPolicy(filerAddress string, name string, policy *PolicyDocument) error {
if !e.initialized { if !e.initialized {

View File

@@ -270,6 +270,9 @@ func (s *STSService) Initialize(config *STSConfig) error {
return fmt.Errorf(ErrConfigCannotBeNil) return fmt.Errorf(ErrConfigCannotBeNil)
} }
// Apply defaults before validation
config.ApplyDefaults()
if err := s.validateConfig(config); err != nil { if err := s.validateConfig(config); err != nil {
return fmt.Errorf("invalid STS configuration: %w", err) return fmt.Errorf("invalid STS configuration: %w", err)
} }
@@ -288,6 +291,21 @@ func (s *STSService) Initialize(config *STSConfig) error {
return nil return nil
} }
// ApplyDefaults applies default values to the STS configuration
func (c *STSConfig) ApplyDefaults() {
if c.TokenDuration.Duration <= 0 {
c.TokenDuration.Duration = time.Duration(DefaultTokenDuration) * time.Second
}
if c.MaxSessionLength.Duration <= 0 {
c.MaxSessionLength.Duration = time.Duration(DefaultMaxSessionLength) * time.Second
}
if c.Issuer == "" {
c.Issuer = DefaultIssuer
}
}
// validateConfig validates the STS configuration // validateConfig validates the STS configuration
func (s *STSService) validateConfig(config *STSConfig) error { func (s *STSService) validateConfig(config *STSConfig) error {
if config.TokenDuration.Duration <= 0 { if config.TokenDuration.Duration <= 0 {

View File

@@ -75,11 +75,34 @@ func TestSTSServiceInitialization(t *testing.T) {
} else { } else {
assert.NoError(t, err) assert.NoError(t, err)
assert.True(t, service.IsInitialized()) assert.True(t, service.IsInitialized())
// Verify defaults if applicable
if tt.config.Issuer == "" {
assert.Equal(t, DefaultIssuer, service.Config.Issuer)
}
if tt.config.TokenDuration.Duration == 0 {
assert.Equal(t, time.Duration(DefaultTokenDuration)*time.Second, service.Config.TokenDuration.Duration)
}
} }
}) })
} }
} }
func TestSTSServiceDefaults(t *testing.T) {
service := NewSTSService()
config := &STSConfig{
SigningKey: []byte("test-signing-key"),
// Missing duration and issuer
}
err := service.Initialize(config)
assert.NoError(t, err)
assert.Equal(t, DefaultIssuer, config.Issuer)
assert.Equal(t, time.Duration(DefaultTokenDuration)*time.Second, config.TokenDuration.Duration)
assert.Equal(t, time.Duration(DefaultMaxSessionLength)*time.Second, config.MaxSessionLength.Duration)
}
// TestAssumeRoleWithWebIdentity tests role assumption with OIDC tokens // TestAssumeRoleWithWebIdentity tests role assumption with OIDC tokens
func TestAssumeRoleWithWebIdentity(t *testing.T) { func TestAssumeRoleWithWebIdentity(t *testing.T) {
service := setupTestSTSService(t) service := setupTestSTSService(t)

View File

@@ -89,7 +89,9 @@ func NewIamApiServerWithStore(router *mux.Router, option *IamServerOption, expli
GrpcDialOption: option.GrpcDialOption, GrpcDialOption: option.GrpcDialOption,
} }
iam := s3api.NewIdentityAccessManagementWithStore(&s3Option, explicitStore) // Initialize FilerClient for IAM - explicit filers only (no discovery as FilerGroup unspecified)
filerClient := wdclient.NewFilerClient(option.Filers, option.GrpcDialOption, "")
iam := s3api.NewIdentityAccessManagementWithStore(&s3Option, filerClient, explicitStore)
configure.credentialManager = iam.GetCredentialManager() configure.credentialManager = iam.GetCredentialManager()
iamApiServer = &IamApiServer{ iamApiServer = &IamApiServer{
@@ -100,6 +102,13 @@ func NewIamApiServerWithStore(router *mux.Router, option *IamServerOption, expli
masterClient: masterClient, masterClient: masterClient,
} }
// Keep attempting to load configuration from filer now that we have a client
go func() {
if err := iam.LoadS3ApiConfigurationFromCredentialManager(); err != nil {
glog.Warningf("Failed to load IAM config from credential manager after client update: %v", err)
}
}()
iamApiServer.registerRouter(router) iamApiServer.registerRouter(router)
return iamApiServer, nil return iamApiServer, nil

View File

@@ -22,6 +22,7 @@ import (
"github.com/seaweedfs/seaweedfs/weed/s3api/policy_engine" "github.com/seaweedfs/seaweedfs/weed/s3api/policy_engine"
"github.com/seaweedfs/seaweedfs/weed/s3api/s3_constants" "github.com/seaweedfs/seaweedfs/weed/s3api/s3_constants"
"github.com/seaweedfs/seaweedfs/weed/s3api/s3err" "github.com/seaweedfs/seaweedfs/weed/s3api/s3err"
"github.com/seaweedfs/seaweedfs/weed/wdclient"
// Import KMS providers to register them // Import KMS providers to register them
_ "github.com/seaweedfs/seaweedfs/weed/kms/aws" _ "github.com/seaweedfs/seaweedfs/weed/kms/aws"
@@ -54,7 +55,7 @@ type IdentityAccessManagement struct {
domain string domain string
isAuthEnabled bool isAuthEnabled bool
credentialManager *credential.CredentialManager credentialManager *credential.CredentialManager
filerClient filer_pb.SeaweedFilerClient filerClient *wdclient.FilerClient
grpcDialOption grpc.DialOption grpcDialOption grpc.DialOption
// IAM Integration for advanced features // IAM Integration for advanced features
@@ -132,15 +133,37 @@ func (c *Credential) isCredentialExpired() bool {
return c.Expiration > 0 && c.Expiration < time.Now().Unix() return c.Expiration > 0 && c.Expiration < time.Now().Unix()
} }
func NewIdentityAccessManagement(option *S3ApiServerOption) *IdentityAccessManagement { // NewIdentityAccessManagement creates a new IAM manager
return NewIdentityAccessManagementWithStore(option, "") // SetFilerClient updates the filer client and its associated credential store
func (iam *IdentityAccessManagement) SetFilerClient(filerClient *wdclient.FilerClient) {
iam.m.Lock()
iam.filerClient = filerClient
iam.m.Unlock()
if iam.credentialManager == nil || filerClient == nil {
return
}
// Update credential store to use FilerClient's current filer for HA
if store := iam.credentialManager.GetStore(); store != nil {
if filerFuncSetter, ok := store.(interface {
SetFilerAddressFunc(func() pb.ServerAddress, grpc.DialOption)
}); ok {
filerFuncSetter.SetFilerAddressFunc(filerClient.GetCurrentFiler, iam.grpcDialOption)
}
}
} }
func NewIdentityAccessManagementWithStore(option *S3ApiServerOption, explicitStore string) *IdentityAccessManagement { func NewIdentityAccessManagement(option *S3ApiServerOption, filerClient *wdclient.FilerClient) *IdentityAccessManagement {
return NewIdentityAccessManagementWithStore(option, filerClient, "")
}
func NewIdentityAccessManagementWithStore(option *S3ApiServerOption, filerClient *wdclient.FilerClient, explicitStore string) *IdentityAccessManagement {
iam := &IdentityAccessManagement{ iam := &IdentityAccessManagement{
domain: option.DomainName, domain: option.DomainName,
hashes: make(map[string]*sync.Pool), hashes: make(map[string]*sync.Pool),
hashCounters: make(map[string]*int32), hashCounters: make(map[string]*int32),
filerClient: filerClient,
} }
// Always initialize credential manager with fallback to defaults // Always initialize credential manager with fallback to defaults
@@ -172,6 +195,25 @@ func NewIdentityAccessManagementWithStore(option *S3ApiServerOption, explicitSto
iam.credentialManager = credentialManager iam.credentialManager = credentialManager
iam.stopChan = make(chan struct{}) iam.stopChan = make(chan struct{})
iam.grpcDialOption = option.GrpcDialOption
// Initialize default anonymous identity
// This ensures consistent behavior for anonymous access:
// 1. In simple auth mode (no IAM integration):
// - lookupAnonymous returns this identity
// - VerifyActionPermission checks actions (which are empty) -> Denies access
// - This preserves the secure-by-default behavior for simple auth
// 2. In advanced IAM mode (with Policy Engine):
// - lookupAnonymous returns this identity
// - VerifyActionPermission proceeds to Policy Engine
// - Policy Engine evaluates against policies (DefaultEffect=Allow if no config)
// - This enables the flexible "Open by Default" for zero-config startup
iam.identityAnonymous = &Identity{
Name: "anonymous",
Account: &AccountAnonymous,
Actions: []Action{},
IsStatic: true,
}
// First, try to load configurations from file or filer // First, try to load configurations from file or filer
startConfigFile := option.Config startConfigFile := option.Config
@@ -552,6 +594,16 @@ func (iam *IdentityAccessManagement) ReplaceS3ApiConfiguration(config *iam_pb.S3
} }
} }
// Ensure anonymous identity exists
if identityAnonymous == nil {
identityAnonymous = &Identity{
Name: "anonymous",
Account: accounts[AccountAnonymous.Id],
Actions: []Action{},
IsStatic: true,
}
}
// atomically switch // atomically switch
iam.identities = identities iam.identities = identities
iam.identityAnonymous = identityAnonymous iam.identityAnonymous = identityAnonymous
@@ -572,6 +624,9 @@ func (iam *IdentityAccessManagement) ReplaceS3ApiConfiguration(config *iam_pb.S3
} }
} }
if !exists { if !exists {
if len(envIdent.Credentials) == 0 {
continue
}
iam.identities = append(iam.identities, envIdent) iam.identities = append(iam.identities, envIdent)
iam.accessKeyIdent[envIdent.Credentials[0].AccessKey] = envIdent iam.accessKeyIdent[envIdent.Credentials[0].AccessKey] = envIdent
iam.nameToIdentity[envIdent.Name] = envIdent iam.nameToIdentity[envIdent.Name] = envIdent
@@ -992,7 +1047,8 @@ func (iam *IdentityAccessManagement) LookupByAccessKey(accessKey string) (identi
return iam.lookupByAccessKey(accessKey) return iam.lookupByAccessKey(accessKey)
} }
func (iam *IdentityAccessManagement) lookupAnonymous() (identity *Identity, found bool) { // LookupAnonymous returns the anonymous identity if it exists
func (iam *IdentityAccessManagement) LookupAnonymous() (identity *Identity, found bool) {
iam.m.RLock() iam.m.RLock()
defer iam.m.RUnlock() defer iam.m.RUnlock()
if iam.identityAnonymous != nil { if iam.identityAnonymous != nil {
@@ -1112,6 +1168,9 @@ func (iam *IdentityAccessManagement) handleAuthResult(w http.ResponseWriter, r *
// Wrapper to maintain backward compatibility // Wrapper to maintain backward compatibility
func (iam *IdentityAccessManagement) authRequest(r *http.Request, action Action) (*Identity, s3err.ErrorCode) { func (iam *IdentityAccessManagement) authRequest(r *http.Request, action Action) (*Identity, s3err.ErrorCode) {
identity, err, _ := iam.authRequestWithAuthType(r, action) identity, err, _ := iam.authRequestWithAuthType(r, action)
if err != s3err.ErrNone {
return nil, err
}
return identity, err return identity, err
} }
@@ -1173,7 +1232,7 @@ func (iam *IdentityAccessManagement) authenticateRequestInternal(r *http.Request
} }
case authTypeAnonymous: case authTypeAnonymous:
amzAuthType = "Anonymous" amzAuthType = "Anonymous"
if identity, found = iam.lookupAnonymous(); !found { if identity, found = iam.LookupAnonymous(); !found {
r.Header.Set(s3_constants.AmzAuthType, amzAuthType) r.Header.Set(s3_constants.AmzAuthType, amzAuthType)
return identity, s3err.ErrAccessDenied, reqAuthType return identity, s3err.ErrAccessDenied, reqAuthType
} }
@@ -1212,8 +1271,8 @@ func (iam *IdentityAccessManagement) authRequestWithAuthType(r *http.Request, ac
// through buckets and checking permissions for each. Skip the global check here. // through buckets and checking permissions for each. Skip the global check here.
policyAllows := false policyAllows := false
if action == s3_constants.ACTION_LIST && bucket == "" { if action == s3_constants.ACTION_LIST && bucket == "" && identity.Name != s3_constants.AccountAnonymousId {
// ListBuckets operation - authorization handled per-bucket in the handler // ListBuckets operation for authenticated users - authorization handled per-bucket in the handler
} else { } else {
// First check bucket policy if one exists // First check bucket policy if one exists
// Bucket policies can grant or deny access to specific users/principals // Bucket policies can grant or deny access to specific users/principals
@@ -1307,8 +1366,8 @@ func (iam *IdentityAccessManagement) AuthSignatureOnly(r *http.Request) (*Identi
return identity, s3err.ErrNotImplemented return identity, s3err.ErrNotImplemented
} }
case authTypeAnonymous: case authTypeAnonymous:
// Anonymous users cannot use IAM API // Anonymous users can be authenticated, but authorization is handled separately
return identity, s3err.ErrAccessDenied return iam.identityAnonymous, s3err.ErrNone
default: default:
return identity, s3err.ErrNotImplemented return identity, s3err.ErrNotImplemented
} }

View File

@@ -450,7 +450,7 @@ func TestNewIdentityAccessManagementWithStoreEnvVars(t *testing.T) {
option := &S3ApiServerOption{ option := &S3ApiServerOption{
Config: "", // No config file - this should trigger environment variable fallback Config: "", // No config file - this should trigger environment variable fallback
} }
iam := NewIdentityAccessManagementWithStore(option, string(credential.StoreTypeMemory)) iam := NewIdentityAccessManagementWithStore(option, nil, string(credential.StoreTypeMemory))
if tt.expectEnvIdentity { if tt.expectEnvIdentity {
// Should have exactly one identity from environment variables // Should have exactly one identity from environment variables
@@ -510,7 +510,7 @@ func TestConfigFileWithNoIdentitiesAllowsEnvVars(t *testing.T) {
option := &S3ApiServerOption{ option := &S3ApiServerOption{
Config: tmpFile.Name(), Config: tmpFile.Name(),
} }
iam := NewIdentityAccessManagementWithStore(option, string(credential.StoreTypeMemory)) iam := NewIdentityAccessManagementWithStore(option, nil, string(credential.StoreTypeMemory))
// Should have exactly one identity from environment variables // Should have exactly one identity from environment variables
assert.Len(t, iam.identities, 1, "Should have exactly one identity from environment variables even when config file exists with no identities") assert.Len(t, iam.identities, 1, "Should have exactly one identity from environment variables even when config file exists with no identities")
@@ -762,7 +762,7 @@ func TestSignatureVerificationDoesNotCheckPermissions(t *testing.T) {
} }
func TestStaticIdentityProtection(t *testing.T) { func TestStaticIdentityProtection(t *testing.T) {
iam := NewIdentityAccessManagement(&S3ApiServerOption{}) iam := NewIdentityAccessManagement(&S3ApiServerOption{}, nil)
// Add a static identity // Add a static identity
staticIdent := &Identity{ staticIdent := &Identity{

View File

@@ -66,7 +66,7 @@ func TestReproIssue7912(t *testing.T) {
option := &S3ApiServerOption{ option := &S3ApiServerOption{
Config: tmpFile.Name(), Config: tmpFile.Name(),
} }
iam := NewIdentityAccessManagementWithStore(option, "memory") iam := NewIdentityAccessManagementWithStore(option, nil, "memory")
assert.True(t, iam.isEnabled(), "Auth should be enabled") assert.True(t, iam.isEnabled(), "Auth should be enabled")

View File

@@ -44,6 +44,10 @@ func (m *MockIAMIntegration) ValidateTrustPolicyForPrincipal(ctx context.Context
return nil return nil
} }
func (m *MockIAMIntegration) DefaultAllow() bool {
return true
}
// TestVerifyV4SignatureWithSTSIdentity tests that verifyV4Signature properly handles STS identities // TestVerifyV4SignatureWithSTSIdentity tests that verifyV4Signature properly handles STS identities
// by falling back to IAM authorization when shouldCheckPermissions is true // by falling back to IAM authorization when shouldCheckPermissions is true
func TestVerifyV4SignatureWithSTSIdentity(t *testing.T) { func TestVerifyV4SignatureWithSTSIdentity(t *testing.T) {

View File

@@ -22,7 +22,7 @@ func TestSTSIdentityPolicyNamesPopulation(t *testing.T) {
stsService, config := setupTestSTSService(t) stsService, config := setupTestSTSService(t)
// Create IAM with STS integration // Create IAM with STS integration
iam := NewIdentityAccessManagementWithStore(&S3ApiServerOption{}, "memory") iam := NewIdentityAccessManagementWithStore(&S3ApiServerOption{}, nil, "memory")
s3iam := &S3IAMIntegration{ s3iam := &S3IAMIntegration{
stsService: stsService, stsService: stsService,
} }
@@ -264,7 +264,7 @@ func TestValidateSTSSessionTokenIntegration(t *testing.T) {
stsService, config := setupTestSTSService(t) stsService, config := setupTestSTSService(t)
// Create IAM with STS integration // Create IAM with STS integration
iam := NewIdentityAccessManagementWithStore(&S3ApiServerOption{}, "memory") iam := NewIdentityAccessManagementWithStore(&S3ApiServerOption{}, nil, "memory")
s3iam := &S3IAMIntegration{ s3iam := &S3IAMIntegration{
stsService: stsService, stsService: stsService,
} }
@@ -311,7 +311,7 @@ func TestSTSIdentityClaimsPopulation(t *testing.T) {
stsService, config := setupTestSTSService(t) stsService, config := setupTestSTSService(t)
// Create IAM with STS integration // Create IAM with STS integration
iam := NewIdentityAccessManagementWithStore(&S3ApiServerOption{}, "memory") iam := NewIdentityAccessManagementWithStore(&S3ApiServerOption{}, nil, "memory")
s3iam := &S3IAMIntegration{ s3iam := &S3IAMIntegration{
stsService: stsService, stsService: stsService,
} }

View File

@@ -0,0 +1,156 @@
package s3api
import (
"os"
"path/filepath"
"testing"
"github.com/stretchr/testify/assert"
)
func TestLoadIAMManagerFromConfig_Defaults(t *testing.T) {
// Create a temporary config file with minimal content (just policy)
tmpDir := t.TempDir()
configPath := filepath.Join(tmpDir, "iam_config.json")
configContent := `{
"sts": {
"providers": []
},
"policy": {
"storeType": "memory",
"defaultEffect": "Allow"
}
}`
err := os.WriteFile(configPath, []byte(configContent), 0644)
assert.NoError(t, err)
// dummy filer address provider
filerProvider := func() string { return "localhost:8888" }
defaultSigningKeyProvider := func() string { return "default-secure-signing-key" }
// Load the manager
manager, err := loadIAMManagerFromConfig(configPath, filerProvider, defaultSigningKeyProvider)
assert.NoError(t, err)
assert.NotNil(t, manager)
}
func TestLoadIAMManagerFromConfig_Overrides(t *testing.T) {
// Create a temporary config file with EXPLICIT values
tmpDir := t.TempDir()
configPath := filepath.Join(tmpDir, "iam_config_explicit.json")
configContent := `{
"sts": {
"tokenDuration": "2h",
"maxSessionLength": "24h",
"issuer": "custom-issuer",
"signingKey": "ZXhwbGljaXQtc2lnbmluZy1rZXktMTIzNDU="
},
"policy": {
"storeType": "memory",
"defaultEffect": "Allow"
}
}`
// Base64 encoded "explicit-signing-key-12345" is "ZXhwbGljaXQtc2lnbmluZy1rZXktMTIzNDU="
err := os.WriteFile(configPath, []byte(configContent), 0644)
assert.NoError(t, err)
filerProvider := func() string { return "localhost:8888" }
defaultSigningKeyProvider := func() string { return "default-secure-signing-key" }
// Load
manager, err := loadIAMManagerFromConfig(configPath, filerProvider, defaultSigningKeyProvider)
assert.NoError(t, err)
assert.NotNil(t, manager)
}
func TestLoadIAMManagerFromConfig_PartialDefaults(t *testing.T) {
// Test that partial configs (e.g. providing SigningKey but not Duration) work
tmpDir := t.TempDir()
configPath := filepath.Join(tmpDir, "iam_config_partial.json")
// Signing key provided in JSON, others missing
configContent := `{
"sts": {
"signingKey": "anNvbi1wcm92aWRlZC1rZXktMTIzNDU="
},
"policy": {
"storeType": "memory",
"defaultEffect": "Allow"
}
}`
err := os.WriteFile(configPath, []byte(configContent), 0644)
assert.NoError(t, err)
filerProvider := func() string { return "localhost:8888" }
// Default signing key provided but should be IGNORED because JSON has one
defaultSigningKeyProvider := func() string { return "server-default-key-should-be-ignored" }
manager, err := loadIAMManagerFromConfig(configPath, filerProvider, defaultSigningKeyProvider)
assert.NoError(t, err)
assert.NotNil(t, manager)
}
func TestLoadIAMManagerFromConfig_ExplicitEmptyKey(t *testing.T) {
// Test that if JSON has empty signing key string, it still falls back
tmpDir := t.TempDir()
configPath := filepath.Join(tmpDir, "iam_config_empty_key.json")
// Signing key explicitly empty
configContent := `{
"sts": {
"signingKey": ""
},
"policy": {
"storeType": "memory",
"defaultEffect": "Allow"
}
}`
err := os.WriteFile(configPath, []byte(configContent), 0644)
assert.NoError(t, err)
filerProvider := func() string { return "localhost:8888" }
defaultSigningKeyProvider := func() string { return "fallback-key-should-be-used" }
manager, err := loadIAMManagerFromConfig(configPath, filerProvider, defaultSigningKeyProvider)
assert.NoError(t, err)
assert.NotNil(t, manager)
}
func TestLoadIAMManagerFromConfig_MissingKeyError(t *testing.T) {
// Test that if BOTH keys are empty, it fails with a clear error
tmpDir := t.TempDir()
configPath := filepath.Join(tmpDir, "iam_config_all_empty.json")
// Signing key explicitly empty in JSON
configContent := `{
"sts": {
"signingKey": ""
},
"policy": {
"storeType": "memory",
"defaultEffect": "Allow"
}
}`
err := os.WriteFile(configPath, []byte(configContent), 0644)
assert.NoError(t, err)
filerProvider := func() string { return "localhost:8888" }
defaultSigningKeyProvider := func() string { return "" } // Empty default too
// Ensure no SSE-S3 key interferes (global state in tests is tricky, but let's assume clean state or no mock)
// Ideally we would mock GetSSES3KeyManager().GetMasterKey() but it's a global singleton.
// For this unit test, if the global key manager has no key, it should fail.
_, err = loadIAMManagerFromConfig(configPath, filerProvider, defaultSigningKeyProvider)
// Should return a clear error
assert.Error(t, err)
assert.Contains(t, err.Error(), "no signing key found for STS service")
}

View File

@@ -0,0 +1,50 @@
package s3api
import (
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestLoadIAMManagerFromConfig_OptionalConfig(t *testing.T) {
// Mock dependencies
filerAddressProvider := func() string { return "localhost:8888" }
getFilerSigningKey := func() string { return "test-signing-key" }
// Test Case 1: Empty config path should load defaults
iamManager, err := loadIAMManagerFromConfig("", filerAddressProvider, getFilerSigningKey)
require.NoError(t, err)
require.NotNil(t, iamManager)
// Verify STS Service is initialized with defaults
stsService := iamManager.GetSTSService()
assert.NotNil(t, stsService)
// Verify defaults are applied
// Since we can't easily access the internal config of stsService,
// we rely on the fact that initialization succeeded without error.
// We can also verify that the policy engine uses memory store by default.
// Verify Policy Engine is initialized with defaults (Memory store, Deny effect)
// Again, internal state might be hard to access directly, but successful init implies defaults worked.
}
func TestLoadIAMManagerFromConfig_EmptyConfigWithFallbackKey(t *testing.T) {
// Mock dependencies where getFilerSigningKey returns empty, forcing fallback logic
// Initialize IAM with empty config (should trigger defaults)
// We pass empty string for config file path
option := &S3ApiServerOption{
Config: "",
IamConfig: "",
EnableIam: true,
}
iamManager := NewIdentityAccessManagementWithStore(option, nil, "memory")
// Verify identityAnonymous is initialized
// This confirms the fix for anonymous access in zero-config mode
anonIdentity, found := iamManager.LookupAnonymous()
assert.True(t, found, "Anonymous identity should be found by default")
assert.NotNil(t, anonIdentity, "Anonymous identity should not be nil")
assert.Equal(t, "anonymous", anonIdentity.Name)
}

View File

@@ -18,6 +18,7 @@ type FilerClient interface {
type S3Authenticator interface { type S3Authenticator interface {
AuthenticateRequest(r *http.Request) (string, interface{}, s3err.ErrorCode) AuthenticateRequest(r *http.Request) (string, interface{}, s3err.ErrorCode)
DefaultAllow() bool
} }
// Server implements the Iceberg REST Catalog API. // Server implements the Iceberg REST Catalog API.
@@ -128,20 +129,25 @@ func (s *Server) Auth(handler http.HandlerFunc) http.HandlerFunc {
identityName, identity, errCode := s.authenticator.AuthenticateRequest(r) identityName, identity, errCode := s.authenticator.AuthenticateRequest(r)
if errCode != s3err.ErrNone { if errCode != s3err.ErrNone {
apiErr := s3err.GetAPIError(errCode) // If authentication failed but DefaultAllow is enabled, proceed without identity
errorType := "RESTException" if s.authenticator.DefaultAllow() {
switch apiErr.HTTPStatusCode { glog.V(2).Infof("Iceberg: AuthenticateRequest failed (%v), but DefaultAllow is true, proceeding", errCode)
case http.StatusForbidden: } else {
errorType = "ForbiddenException" apiErr := s3err.GetAPIError(errCode)
case http.StatusUnauthorized: errorType := "RESTException"
errorType = "NotAuthorizedException" switch apiErr.HTTPStatusCode {
case http.StatusBadRequest: case http.StatusForbidden:
errorType = "BadRequestException" errorType = "ForbiddenException"
case http.StatusInternalServerError: case http.StatusUnauthorized:
errorType = "InternalServerError" errorType = "NotAuthorizedException"
case http.StatusBadRequest:
errorType = "BadRequestException"
case http.StatusInternalServerError:
errorType = "InternalServerError"
}
writeError(w, apiErr.HTTPStatusCode, errorType, apiErr.Description)
return
} }
writeError(w, apiErr.HTTPStatusCode, errorType, apiErr.Description)
return
} }
if identityName != "" || identity != nil { if identityName != "" || identity != nil {

View File

@@ -44,6 +44,7 @@ type IAMIntegration interface {
AuthorizeAction(ctx context.Context, identity *IAMIdentity, action Action, bucket string, objectKey string, r *http.Request) s3err.ErrorCode AuthorizeAction(ctx context.Context, identity *IAMIdentity, action Action, bucket string, objectKey string, r *http.Request) s3err.ErrorCode
ValidateSessionToken(ctx context.Context, token string) (*sts.SessionInfo, error) ValidateSessionToken(ctx context.Context, token string) (*sts.SessionInfo, error)
ValidateTrustPolicyForPrincipal(ctx context.Context, roleArn, principalArn string) error ValidateTrustPolicyForPrincipal(ctx context.Context, roleArn, principalArn string) error
DefaultAllow() bool
} }
// S3IAMIntegration provides IAM integration for S3 API // S3IAMIntegration provides IAM integration for S3 API
@@ -310,6 +311,14 @@ func (s3iam *S3IAMIntegration) ValidateTrustPolicyForPrincipal(ctx context.Conte
return s3iam.iamManager.ValidateTrustPolicyForPrincipal(ctx, roleArn, principalArn) return s3iam.iamManager.ValidateTrustPolicyForPrincipal(ctx, roleArn, principalArn)
} }
// DefaultAllow returns whether access is allowed by default when no policy is found
func (s3iam *S3IAMIntegration) DefaultAllow() bool {
if s3iam.iamManager == nil {
return true // Default to true if IAM is not enabled
}
return s3iam.iamManager.DefaultAllow()
}
// IAMIdentity represents an authenticated identity with session information // IAMIdentity represents an authenticated identity with session information
type IAMIdentity struct { type IAMIdentity struct {
Name string Name string

View File

@@ -5,6 +5,7 @@ import (
"crypto/aes" "crypto/aes"
"crypto/cipher" "crypto/cipher"
"crypto/rand" "crypto/rand"
"crypto/sha256"
"encoding/base64" "encoding/base64"
"encoding/hex" "encoding/hex"
"encoding/json" "encoding/json"
@@ -19,9 +20,13 @@ import (
"time" "time"
"github.com/seaweedfs/seaweedfs/weed/glog" "github.com/seaweedfs/seaweedfs/weed/glog"
"github.com/seaweedfs/seaweedfs/weed/pb"
"github.com/seaweedfs/seaweedfs/weed/pb/filer_pb" "github.com/seaweedfs/seaweedfs/weed/pb/filer_pb"
"github.com/seaweedfs/seaweedfs/weed/s3api/s3_constants" "github.com/seaweedfs/seaweedfs/weed/s3api/s3_constants"
"github.com/seaweedfs/seaweedfs/weed/util" "github.com/seaweedfs/seaweedfs/weed/util"
"github.com/seaweedfs/seaweedfs/weed/wdclient"
"golang.org/x/crypto/hkdf"
"google.golang.org/grpc"
) )
// SSE-S3 uses AES-256 encryption with server-managed keys // SSE-S3 uses AES-256 encryption with server-managed keys
@@ -452,6 +457,27 @@ func (km *SSES3KeyManager) GetKey(keyID string) (*SSES3Key, bool) {
return nil, false return nil, false
} }
// GetMasterKey returns a derived key from the master KEK for STS signing
// This uses HKDF to isolate the STS security domain from the SSE-S3 domain
func (km *SSES3KeyManager) GetMasterKey() []byte {
km.mu.RLock()
defer km.mu.RUnlock()
if len(km.superKey) == 0 {
return nil
}
// Derive a separate key for STS to isolate security domains
// We use the KEK as the secret, and "seaweedfs-sts-signing-key" as the info
hkdfReader := hkdf.New(sha256.New, km.superKey, nil, []byte("seaweedfs-sts-signing-key"))
derived := make([]byte, 32) // 256-bit derived key
if _, err := io.ReadFull(hkdfReader, derived); err != nil {
glog.Errorf("Failed to derive STS key: %v", err)
return nil
}
return derived
}
// Global SSE-S3 key manager instance // Global SSE-S3 key manager instance
var globalSSES3KeyManager = NewSSES3KeyManager() var globalSSES3KeyManager = NewSSES3KeyManager()
@@ -460,9 +486,31 @@ func GetSSES3KeyManager() *SSES3KeyManager {
return globalSSES3KeyManager return globalSSES3KeyManager
} }
// KeyManagerFilerClient wraps wdclient.FilerClient to satisfy filer_pb.FilerClient interface
type KeyManagerFilerClient struct {
*wdclient.FilerClient
grpcDialOption grpc.DialOption
}
func (k *KeyManagerFilerClient) AdjustedUrl(location *filer_pb.Location) string {
return location.Url
}
func (k *KeyManagerFilerClient) WithFilerClient(streamingMode bool, fn func(filer_pb.SeaweedFilerClient) error) error {
filerAddress := k.GetCurrentFiler()
if filerAddress == "" {
return fmt.Errorf("no filer available")
}
return pb.WithGrpcFilerClient(streamingMode, 0, filerAddress, k.grpcDialOption, fn)
}
// InitializeGlobalSSES3KeyManager initializes the global key manager with filer access // InitializeGlobalSSES3KeyManager initializes the global key manager with filer access
func InitializeGlobalSSES3KeyManager(s3ApiServer *S3ApiServer) error { func InitializeGlobalSSES3KeyManager(filerClient *wdclient.FilerClient, grpcDialOption grpc.DialOption) error {
return globalSSES3KeyManager.InitializeWithFiler(s3ApiServer) wrapper := &KeyManagerFilerClient{
FilerClient: filerClient,
grpcDialOption: grpcDialOption,
}
return globalSSES3KeyManager.InitializeWithFiler(wrapper)
} }
// ProcessSSES3Request processes an SSE-S3 request and returns encryption metadata // ProcessSSES3Request processes an SSE-S3 request and returns encryption metadata

View File

@@ -13,7 +13,7 @@ import (
func TestGetRequestDataReader_ChunkedEncodingWithoutIAM(t *testing.T) { func TestGetRequestDataReader_ChunkedEncodingWithoutIAM(t *testing.T) {
// Create an S3ApiServer with IAM disabled // Create an S3ApiServer with IAM disabled
s3a := &S3ApiServer{ s3a := &S3ApiServer{
iam: NewIdentityAccessManagementWithStore(&S3ApiServerOption{}, string(credential.StoreTypeMemory)), iam: NewIdentityAccessManagementWithStore(&S3ApiServerOption{}, nil, string(credential.StoreTypeMemory)),
} }
// Ensure IAM is disabled for this test // Ensure IAM is disabled for this test
s3a.iam.isAuthEnabled = false s3a.iam.isAuthEnabled = false
@@ -87,7 +87,7 @@ func TestGetRequestDataReader_ChunkedEncodingWithoutIAM(t *testing.T) {
func TestGetRequestDataReader_AuthTypeDetection(t *testing.T) { func TestGetRequestDataReader_AuthTypeDetection(t *testing.T) {
// Create an S3ApiServer with IAM disabled // Create an S3ApiServer with IAM disabled
s3a := &S3ApiServer{ s3a := &S3ApiServer{
iam: NewIdentityAccessManagementWithStore(&S3ApiServerOption{}, string(credential.StoreTypeMemory)), iam: NewIdentityAccessManagementWithStore(&S3ApiServerOption{}, nil, string(credential.StoreTypeMemory)),
} }
s3a.iam.isAuthEnabled = false s3a.iam.isAuthEnabled = false
@@ -122,7 +122,7 @@ func TestGetRequestDataReader_AuthTypeDetection(t *testing.T) {
func TestGetRequestDataReader_IAMEnabled(t *testing.T) { func TestGetRequestDataReader_IAMEnabled(t *testing.T) {
// Create an S3ApiServer with IAM enabled // Create an S3ApiServer with IAM enabled
s3a := &S3ApiServer{ s3a := &S3ApiServer{
iam: NewIdentityAccessManagementWithStore(&S3ApiServerOption{}, string(credential.StoreTypeMemory)), iam: NewIdentityAccessManagementWithStore(&S3ApiServerOption{}, nil, string(credential.StoreTypeMemory)),
} }
s3a.iam.isAuthEnabled = true s3a.iam.isAuthEnabled = true

View File

@@ -110,7 +110,8 @@ func NewS3ApiServerWithStore(router *mux.Router, option *S3ApiServerOption, expl
option.AllowedOrigins = domains option.AllowedOrigins = domains
} }
iam := NewIdentityAccessManagementWithStore(option, explicitStore) // Initialize basic/legacy IAM - filerClient not available yet, passed as nil
iam := NewIdentityAccessManagementWithStore(option, nil, explicitStore)
// Initialize bucket policy engine first // Initialize bucket policy engine first
policyEngine := NewBucketPolicyEngine() policyEngine := NewBucketPolicyEngine()
@@ -146,17 +147,25 @@ func NewS3ApiServerWithStore(router *mux.Router, option *S3ApiServerOption, expl
glog.V(1).Infof("S3 API initialized FilerClient with %d filer(s) (no discovery)", len(option.Filers)) glog.V(1).Infof("S3 API initialized FilerClient with %d filer(s) (no discovery)", len(option.Filers))
} }
// Update credential store to use FilerClient's current filer for HA // Initialize Global SSE-S3 Key Manager early so it's available for IAM fallback
if store := iam.credentialManager.GetStore(); store != nil { // This ensures we can access the KEK for STS signing key if needed
if filerFuncSetter, ok := store.(interface { if err := InitializeGlobalSSES3KeyManager(filerClient, option.GrpcDialOption); err != nil {
SetFilerAddressFunc(func() pb.ServerAddress, grpc.DialOption) glog.Errorf("Failed to initialize SSE-S3 Key Manager: %v", err)
}); ok { // We continue, as this might be a transient failure or non-critical for some setups,
// Use FilerClient's GetCurrentFiler for true HA // but IAM fallback to KEK will fail if this didn't succeed.
filerFuncSetter.SetFilerAddressFunc(filerClient.GetCurrentFiler, option.GrpcDialOption)
glog.V(1).Infof("Updated credential store to use FilerClient's current active filer (HA-aware)")
}
} }
// Update credential store to use FilerClient's current filer for HA
iam.SetFilerClient(filerClient)
// Keep attempting to load configuration from filer now that we have a client
// The initial load in NewIdentityAccessManagementWithStore might have failed if client was nil
go func() {
if err := iam.loadS3ApiConfigurationFromFiler(option); err != nil {
glog.Warningf("Failed to load IAM config from filer after client update: %v", err)
}
}()
s3ApiServer = &S3ApiServer{ s3ApiServer = &S3ApiServer{
option: option, option: option,
iam: iam, iam: iam,
@@ -178,19 +187,25 @@ func NewS3ApiServerWithStore(router *mux.Router, option *S3ApiServerOption, expl
// This avoids circular dependency by not passing the entire S3ApiServer // This avoids circular dependency by not passing the entire S3ApiServer
iam.policyEngine = policyEngine iam.policyEngine = policyEngine
// Initialize advanced IAM system if config is provided // Initialize advanced IAM system if config is provided or explicitly enabled
if option.IamConfig != "" { if option.IamConfig != "" || option.EnableIam {
glog.V(1).Infof("Loading advanced IAM configuration from: %s", option.IamConfig) configSource := "defaults"
if option.IamConfig != "" {
configSource = option.IamConfig
}
glog.V(1).Infof("Loading advanced IAM configuration from: %s", configSource)
// Use FilerClient's GetCurrentFiler for HA-aware filer selection // Use FilerClient's GetCurrentFiler for HA-aware filer selection
iamManager, err := loadIAMManagerFromConfig(option.IamConfig, func() string { iamManager, err := loadIAMManagerFromConfig(option.IamConfig, func() string {
return string(filerClient.GetCurrentFiler()) return string(filerClient.GetCurrentFiler())
}, func() string {
return signingKey
}) })
if err != nil { if err != nil {
glog.Errorf("Failed to load IAM configuration: %v", err) glog.Errorf("Failed to load IAM configuration: %v", err)
} else { } else {
if iam.credentialManager != nil { if s3ApiServer.iam.credentialManager != nil {
iamManager.SetUserStore(iam.credentialManager) iamManager.SetUserStore(s3ApiServer.iam.credentialManager)
} }
glog.V(1).Infof("IAM Manager loaded, creating integration") glog.V(1).Infof("IAM Manager loaded, creating integration")
// Create S3 IAM integration with the loaded IAM manager // Create S3 IAM integration with the loaded IAM manager
@@ -233,6 +248,10 @@ func NewS3ApiServerWithStore(router *mux.Router, option *S3ApiServerOption, expl
}) })
} }
s3ApiServer.bucketRegistry = NewBucketRegistry(s3ApiServer) s3ApiServer.bucketRegistry = NewBucketRegistry(s3ApiServer)
// Update IAM with the final filer client (already handled by SetFilerClient above,
// but this reinforces it if we ever change the flow)
s3ApiServer.iam.SetFilerClient(s3ApiServer.filerClient)
if option.LocalFilerSocket == "" { if option.LocalFilerSocket == "" {
if s3ApiServer.client, err = util_http.NewGlobalHttpClient(); err != nil { if s3ApiServer.client, err = util_http.NewGlobalHttpClient(); err != nil {
return nil, err return nil, err
@@ -249,11 +268,6 @@ func NewS3ApiServerWithStore(router *mux.Router, option *S3ApiServerOption, expl
s3ApiServer.registerRouter(router) s3ApiServer.registerRouter(router)
// Initialize the global SSE-S3 key manager with filer access
if err := InitializeGlobalSSES3KeyManager(s3ApiServer); err != nil {
return nil, fmt.Errorf("failed to initialize SSE-S3 key manager: %w", err)
}
go s3ApiServer.subscribeMetaEvents("s3", startTsNs, filer.DirectoryEtcRoot, []string{ go s3ApiServer.subscribeMetaEvents("s3", startTsNs, filer.DirectoryEtcRoot, []string{
option.BucketsPath, option.BucketsPath,
filer.IamConfigDirectory, filer.IamConfigDirectory,
@@ -830,14 +844,7 @@ func (s3a *S3ApiServer) registerRouter(router *mux.Router) {
} }
// loadIAMManagerFromConfig loads the advanced IAM manager from configuration file // loadIAMManagerFromConfig loads the advanced IAM manager from configuration file
func loadIAMManagerFromConfig(configPath string, filerAddressProvider func() string) (*integration.IAMManager, error) { func loadIAMManagerFromConfig(configPath string, filerAddressProvider func() string, getFilerSigningKey func() string) (*integration.IAMManager, error) {
// Read configuration file
configData, err := os.ReadFile(configPath)
if err != nil {
return nil, fmt.Errorf("failed to read config file: %w", err)
}
// Parse configuration structure
var configRoot struct { var configRoot struct {
STS *sts.STSConfig `json:"sts"` STS *sts.STSConfig `json:"sts"`
Policy *policy.PolicyEngineConfig `json:"policy"` Policy *policy.PolicyEngineConfig `json:"policy"`
@@ -849,24 +856,43 @@ func loadIAMManagerFromConfig(configPath string, filerAddressProvider func() str
} `json:"policies"` } `json:"policies"`
} }
if err := json.Unmarshal(configData, &configRoot); err != nil { if configPath != "" {
return nil, fmt.Errorf("failed to parse config: %w", err) // Read configuration file
configData, err := os.ReadFile(configPath)
if err != nil {
return nil, fmt.Errorf("failed to read config file: %w", err)
}
if err := json.Unmarshal(configData, &configRoot); err != nil {
return nil, fmt.Errorf("failed to parse config: %w", err)
}
} else {
glog.V(1).Infof("No IAM config file provided; using defaults")
// Initialize with empty config which will trigger defaults below
}
// Ensure STS config exists so we can apply defaults later
if configRoot.STS == nil {
configRoot.STS = &sts.STSConfig{}
} }
// Ensure a valid policy engine config exists // Ensure a valid policy engine config exists
if configRoot.Policy == nil { if configRoot.Policy == nil {
// Provide a secure default if not specified in the config file configRoot.Policy = &policy.PolicyEngineConfig{}
// Default to Deny with in-memory store so that JSON-defined policies work without filer }
glog.V(1).Infof("No policy engine config provided; using defaults (DefaultEffect=%s, StoreType=%s)", sts.EffectDeny, sts.StoreTypeMemory) if configRoot.Policy.StoreType == "" {
configRoot.Policy = &policy.PolicyEngineConfig{
DefaultEffect: sts.EffectDeny,
StoreType: sts.StoreTypeMemory,
}
} else if configRoot.Policy.StoreType == "" {
// If policy config exists but storeType is not specified, use memory store
// This ensures JSON-defined policies are stored in memory and work correctly
configRoot.Policy.StoreType = sts.StoreTypeMemory configRoot.Policy.StoreType = sts.StoreTypeMemory
glog.V(1).Infof("Policy storeType not specified; using memory store for JSON config-based setup") }
if configRoot.Policy.DefaultEffect == "" {
// Default to Allow (open) with in-memory store so that
// users can start using STS without locking themselves out immediately.
// For other stores (e.g. filer), default to Deny (closed) for security.
if configRoot.Policy.StoreType == sts.StoreTypeMemory {
configRoot.Policy.DefaultEffect = sts.EffectAllow
} else {
configRoot.Policy.DefaultEffect = sts.EffectDeny
}
glog.V(1).Infof("Using policy defaults: DefaultEffect=%s, StoreType=%s", configRoot.Policy.DefaultEffect, configRoot.Policy.StoreType)
} }
// Create IAM configuration // Create IAM configuration
@@ -878,6 +904,26 @@ func loadIAMManagerFromConfig(configPath string, filerAddressProvider func() str
}, },
} }
// Apply default signing key if not present in config
if iamConfig.STS != nil && len(iamConfig.STS.SigningKey) == 0 {
// 1. Try server-configured signing key (security.toml / CLI)
if key := getFilerSigningKey(); key != "" {
iamConfig.STS.SigningKey = []byte(key)
glog.V(1).Infof("Using default filer signing key for STS service")
} else {
// 2. Try cluster-wide SSE-S3 Master Key (KEK) from Filer
// This ensures zero-config consistency across the cluster
if kek := GetSSES3KeyManager().GetMasterKey(); len(kek) > 0 {
iamConfig.STS.SigningKey = kek
glog.V(1).Infof("Using SSE-S3 Master Key (KEK) for STS service")
} else {
// 3. Fail if no signing key is available
// This ensures consistency across multiple S3 servers and secure operation
return nil, fmt.Errorf("no signing key found for STS service; please provide 'signingKey' in IAM config, configure 'jwt.filer_signing.key' in security.toml, or ensure SSE-S3 is initialized")
}
}
}
// Initialize IAM manager // Initialize IAM manager
iamManager := integration.NewIAMManager() iamManager := integration.NewIAMManager()
if err := iamManager.Initialize(iamConfig, filerAddressProvider); err != nil { if err := iamManager.Initialize(iamConfig, filerAddressProvider); err != nil {
@@ -960,3 +1006,11 @@ func (s3a *S3ApiServer) AuthenticateRequest(r *http.Request) (string, interface{
} }
return "", nil, err return "", nil, err
} }
// DefaultAllow returns whether access is allowed by default when no policy is found
func (s3a *S3ApiServer) DefaultAllow() bool {
if s3a.iam == nil || s3a.iam.iamIntegration == nil {
return false
}
return s3a.iam.iamIntegration.DefaultAllow()
}

View File

@@ -16,7 +16,7 @@ import (
// setupRoutingTestServer creates a minimal S3ApiServer for routing tests // setupRoutingTestServer creates a minimal S3ApiServer for routing tests
func setupRoutingTestServer(t *testing.T) *S3ApiServer { func setupRoutingTestServer(t *testing.T) *S3ApiServer {
opt := &S3ApiServerOption{EnableIam: true} opt := &S3ApiServerOption{EnableIam: true}
iam := NewIdentityAccessManagementWithStore(opt, "memory") iam := NewIdentityAccessManagementWithStore(opt, nil, "memory")
iam.isAuthEnabled = true iam.isAuthEnabled = true
if iam.credentialManager == nil { if iam.credentialManager == nil {

View File

@@ -43,6 +43,11 @@ func (st *S3TablesApiServer) SetAccountID(accountID string) {
st.handler.SetAccountID(accountID) st.handler.SetAccountID(accountID)
} }
// SetDefaultAllow sets whether to allow access by default
func (st *S3TablesApiServer) SetDefaultAllow(allow bool) {
st.handler.SetDefaultAllow(allow)
}
// S3TablesHandler handles S3 Tables API requests // S3TablesHandler handles S3 Tables API requests
func (st *S3TablesApiServer) S3TablesHandler(w http.ResponseWriter, r *http.Request) { func (st *S3TablesApiServer) S3TablesHandler(w http.ResponseWriter, r *http.Request) {
st.handler.HandleRequest(w, r, st) st.handler.HandleRequest(w, r, st)
@@ -57,6 +62,12 @@ func (st *S3TablesApiServer) WithFilerClient(streamingMode bool, fn func(filer_p
func (s3a *S3ApiServer) registerS3TablesRoutes(router *mux.Router) { func (s3a *S3ApiServer) registerS3TablesRoutes(router *mux.Router) {
// Create S3 Tables handler // Create S3 Tables handler
s3TablesApi := NewS3TablesApiServer(s3a) s3TablesApi := NewS3TablesApiServer(s3a)
if s3a.iam != nil && s3a.iam.iamIntegration != nil {
s3TablesApi.SetDefaultAllow(s3a.iam.iamIntegration.DefaultAllow())
} else {
// If IAM is not configured, allow all access by default
s3TablesApi.SetDefaultAllow(true)
}
// Regex for S3 Tables Bucket ARN // Regex for S3 Tables Bucket ARN
const tableBucketARNRegex = "arn:aws:s3tables:[^/:]*:[^/:]*:bucket/[^/]+" const tableBucketARNRegex = "arn:aws:s3tables:[^/:]*:[^/:]*:bucket/[^/]+"
@@ -618,9 +629,15 @@ func (s3a *S3ApiServer) authenticateS3Tables(f http.HandlerFunc) http.HandlerFun
// Use AuthSignatureOnly to authenticate the request without authorizing specific actions // Use AuthSignatureOnly to authenticate the request without authorizing specific actions
identity, errCode := s3a.iam.AuthSignatureOnly(r) identity, errCode := s3a.iam.AuthSignatureOnly(r)
if errCode != s3err.ErrNone { if errCode != s3err.ErrNone {
glog.Errorf("S3Tables: AuthSignatureOnly failed: %v", errCode) // If IAM is enabled but DefaultAllow is true, we can proceed even if unauthenticated
s3err.WriteErrorResponse(w, r, errCode) // authorization checks in handlers will then use DefaultAllow logic.
return if s3a.iam.iamIntegration != nil && s3a.iam.iamIntegration.DefaultAllow() {
glog.V(2).Infof("S3Tables: AuthSignatureOnly failed (%v), but DefaultAllow is true, proceeding", errCode)
} else {
glog.Errorf("S3Tables: AuthSignatureOnly failed: %v", errCode)
s3err.WriteErrorResponse(w, r, errCode)
return
}
} }
// Store the authenticated identity in request context // Store the authenticated identity in request context

View File

@@ -44,15 +44,17 @@ const (
// S3TablesHandler handles S3 Tables API requests // S3TablesHandler handles S3 Tables API requests
type S3TablesHandler struct { type S3TablesHandler struct {
region string region string
accountID string accountID string
defaultAllow bool // Whether to allow access by default (for zero-config IAM)
} }
// NewS3TablesHandler creates a new S3 Tables handler // NewS3TablesHandler creates a new S3 Tables handler
func NewS3TablesHandler() *S3TablesHandler { func NewS3TablesHandler() *S3TablesHandler {
return &S3TablesHandler{ return &S3TablesHandler{
region: DefaultRegion, region: DefaultRegion,
accountID: DefaultAccountID, accountID: DefaultAccountID,
defaultAllow: false,
} }
} }
@@ -70,6 +72,11 @@ func (h *S3TablesHandler) SetAccountID(accountID string) {
} }
} }
// SetDefaultAllow sets whether to allow access by default
func (h *S3TablesHandler) SetDefaultAllow(allow bool) {
h.defaultAllow = allow
}
// FilerClient interface for filer operations // FilerClient interface for filer operations
type FilerClient interface { type FilerClient interface {
WithFilerClient(streamingMode bool, fn func(client filer_pb.SeaweedFilerClient) error) error WithFilerClient(streamingMode bool, fn func(client filer_pb.SeaweedFilerClient) error) error

View File

@@ -16,7 +16,9 @@ import (
func (h *S3TablesHandler) handleCreateTableBucket(w http.ResponseWriter, r *http.Request, filerClient FilerClient) error { func (h *S3TablesHandler) handleCreateTableBucket(w http.ResponseWriter, r *http.Request, filerClient FilerClient) error {
// Check permission // Check permission
principal := h.getAccountID(r) principal := h.getAccountID(r)
if !CanCreateTableBucket(principal, principal, "") { if !CheckPermissionWithContext("CreateTableBucket", principal, principal, "", "", &PolicyContext{
DefaultAllow: h.defaultAllow,
}) {
h.writeError(w, http.StatusForbidden, ErrCodeAccessDenied, "not authorized to create table buckets") h.writeError(w, http.StatusForbidden, ErrCodeAccessDenied, "not authorized to create table buckets")
return NewAuthError("CreateTableBucket", principal, "not authorized to create table buckets") return NewAuthError("CreateTableBucket", principal, "not authorized to create table buckets")
} }

View File

@@ -72,6 +72,7 @@ func (h *S3TablesHandler) handleGetTableBucket(w http.ResponseWriter, r *http.Re
if !CheckPermissionWithContext("GetTableBucket", principal, metadata.OwnerAccountID, bucketPolicy, bucketARN, &PolicyContext{ if !CheckPermissionWithContext("GetTableBucket", principal, metadata.OwnerAccountID, bucketPolicy, bucketARN, &PolicyContext{
TableBucketName: bucketName, TableBucketName: bucketName,
IdentityActions: identityActions, IdentityActions: identityActions,
DefaultAllow: h.defaultAllow,
}) { }) {
h.writeError(w, http.StatusForbidden, ErrCodeAccessDenied, "not authorized to get table bucket details") h.writeError(w, http.StatusForbidden, ErrCodeAccessDenied, "not authorized to get table bucket details")
return ErrAccessDenied return ErrAccessDenied
@@ -101,6 +102,7 @@ func (h *S3TablesHandler) handleListTableBuckets(w http.ResponseWriter, r *http.
identityActions := getIdentityActions(r) identityActions := getIdentityActions(r)
if !CheckPermissionWithContext("ListTableBuckets", principal, accountID, "", "", &PolicyContext{ if !CheckPermissionWithContext("ListTableBuckets", principal, accountID, "", "", &PolicyContext{
IdentityActions: identityActions, IdentityActions: identityActions,
DefaultAllow: h.defaultAllow,
}) { }) {
h.writeError(w, http.StatusForbidden, ErrCodeAccessDenied, "not authorized to list table buckets") h.writeError(w, http.StatusForbidden, ErrCodeAccessDenied, "not authorized to list table buckets")
return NewAuthError("ListTableBuckets", principal, "not authorized to list table buckets") return NewAuthError("ListTableBuckets", principal, "not authorized to list table buckets")
@@ -198,6 +200,7 @@ func (h *S3TablesHandler) handleListTableBuckets(w http.ResponseWriter, r *http.
if !CheckPermissionWithContext("GetTableBucket", accountID, metadata.OwnerAccountID, bucketPolicy, bucketARN, &PolicyContext{ if !CheckPermissionWithContext("GetTableBucket", accountID, metadata.OwnerAccountID, bucketPolicy, bucketARN, &PolicyContext{
TableBucketName: entry.Entry.Name, TableBucketName: entry.Entry.Name,
IdentityActions: identityActions, IdentityActions: identityActions,
DefaultAllow: h.defaultAllow,
}) { }) {
continue continue
} }
@@ -300,6 +303,7 @@ func (h *S3TablesHandler) handleDeleteTableBucket(w http.ResponseWriter, r *http
if !CheckPermissionWithContext("DeleteTableBucket", principal, metadata.OwnerAccountID, bucketPolicy, bucketARN, &PolicyContext{ if !CheckPermissionWithContext("DeleteTableBucket", principal, metadata.OwnerAccountID, bucketPolicy, bucketARN, &PolicyContext{
TableBucketName: bucketName, TableBucketName: bucketName,
IdentityActions: identityActions, IdentityActions: identityActions,
DefaultAllow: h.defaultAllow,
}) { }) {
return NewAuthError("DeleteTableBucket", principal, fmt.Sprintf("not authorized to delete bucket %s", bucketName)) return NewAuthError("DeleteTableBucket", principal, fmt.Sprintf("not authorized to delete bucket %s", bucketName))
} }

View File

@@ -118,6 +118,7 @@ func (h *S3TablesHandler) handleCreateNamespace(w http.ResponseWriter, r *http.R
Namespace: namespaceName, Namespace: namespaceName,
TableBucketTags: bucketTags, TableBucketTags: bucketTags,
IdentityActions: identityActions, IdentityActions: identityActions,
DefaultAllow: h.defaultAllow,
}) { }) {
glog.Infof("S3Tables: Permission denied for CreateNamespace - principal=%s, owner=%s", principal, bucketMetadata.OwnerAccountID) glog.Infof("S3Tables: Permission denied for CreateNamespace - principal=%s, owner=%s", principal, bucketMetadata.OwnerAccountID)
h.writeError(w, http.StatusForbidden, ErrCodeAccessDenied, "not authorized to create namespace in this bucket") h.writeError(w, http.StatusForbidden, ErrCodeAccessDenied, "not authorized to create namespace in this bucket")
@@ -258,6 +259,7 @@ func (h *S3TablesHandler) handleGetNamespace(w http.ResponseWriter, r *http.Requ
Namespace: namespaceName, Namespace: namespaceName,
TableBucketTags: bucketTags, TableBucketTags: bucketTags,
IdentityActions: identityActions, IdentityActions: identityActions,
DefaultAllow: h.defaultAllow,
}) { }) {
h.writeError(w, http.StatusNotFound, ErrCodeNoSuchNamespace, "namespace not found") h.writeError(w, http.StatusNotFound, ErrCodeNoSuchNamespace, "namespace not found")
return ErrAccessDenied return ErrAccessDenied
@@ -344,6 +346,7 @@ func (h *S3TablesHandler) handleListNamespaces(w http.ResponseWriter, r *http.Re
TableBucketName: bucketName, TableBucketName: bucketName,
TableBucketTags: bucketTags, TableBucketTags: bucketTags,
IdentityActions: identityActions, IdentityActions: identityActions,
DefaultAllow: h.defaultAllow,
}) { }) {
h.writeError(w, http.StatusNotFound, ErrCodeNoSuchBucket, fmt.Sprintf("table bucket %s not found", bucketName)) h.writeError(w, http.StatusNotFound, ErrCodeNoSuchBucket, fmt.Sprintf("table bucket %s not found", bucketName))
return ErrAccessDenied return ErrAccessDenied
@@ -528,6 +531,7 @@ func (h *S3TablesHandler) handleDeleteNamespace(w http.ResponseWriter, r *http.R
Namespace: namespaceName, Namespace: namespaceName,
TableBucketTags: bucketTags, TableBucketTags: bucketTags,
IdentityActions: identityActions, IdentityActions: identityActions,
DefaultAllow: h.defaultAllow,
}) { }) {
h.writeError(w, http.StatusNotFound, ErrCodeNoSuchNamespace, "namespace not found") h.writeError(w, http.StatusNotFound, ErrCodeNoSuchNamespace, "namespace not found")
return ErrAccessDenied return ErrAccessDenied

View File

@@ -94,6 +94,7 @@ func (h *S3TablesHandler) handlePutTableBucketPolicy(w http.ResponseWriter, r *h
if !CheckPermissionWithContext("PutTableBucketPolicy", principal, bucketMetadata.OwnerAccountID, "", bucketARN, &PolicyContext{ if !CheckPermissionWithContext("PutTableBucketPolicy", principal, bucketMetadata.OwnerAccountID, "", bucketARN, &PolicyContext{
TableBucketName: bucketName, TableBucketName: bucketName,
IdentityActions: identityActions, IdentityActions: identityActions,
DefaultAllow: h.defaultAllow,
}) { }) {
h.writeError(w, http.StatusForbidden, ErrCodeAccessDenied, "not authorized to put table bucket policy") h.writeError(w, http.StatusForbidden, ErrCodeAccessDenied, "not authorized to put table bucket policy")
return NewAuthError("PutTableBucketPolicy", principal, "not authorized to put table bucket policy") return NewAuthError("PutTableBucketPolicy", principal, "not authorized to put table bucket policy")
@@ -171,6 +172,7 @@ func (h *S3TablesHandler) handleGetTableBucketPolicy(w http.ResponseWriter, r *h
if !CheckPermissionWithContext("GetTableBucketPolicy", principal, bucketMetadata.OwnerAccountID, string(policy), bucketARN, &PolicyContext{ if !CheckPermissionWithContext("GetTableBucketPolicy", principal, bucketMetadata.OwnerAccountID, string(policy), bucketARN, &PolicyContext{
TableBucketName: bucketName, TableBucketName: bucketName,
IdentityActions: identityActions, IdentityActions: identityActions,
DefaultAllow: h.defaultAllow,
}) { }) {
h.writeError(w, http.StatusForbidden, ErrCodeAccessDenied, "not authorized to get table bucket policy") h.writeError(w, http.StatusForbidden, ErrCodeAccessDenied, "not authorized to get table bucket policy")
return NewAuthError("GetTableBucketPolicy", principal, "not authorized to get table bucket policy") return NewAuthError("GetTableBucketPolicy", principal, "not authorized to get table bucket policy")
@@ -246,6 +248,7 @@ func (h *S3TablesHandler) handleDeleteTableBucketPolicy(w http.ResponseWriter, r
if !CheckPermissionWithContext("DeleteTableBucketPolicy", principal, bucketMetadata.OwnerAccountID, bucketPolicy, bucketARN, &PolicyContext{ if !CheckPermissionWithContext("DeleteTableBucketPolicy", principal, bucketMetadata.OwnerAccountID, bucketPolicy, bucketARN, &PolicyContext{
TableBucketName: bucketName, TableBucketName: bucketName,
IdentityActions: identityActions, IdentityActions: identityActions,
DefaultAllow: h.defaultAllow,
}) { }) {
h.writeError(w, http.StatusForbidden, ErrCodeAccessDenied, "not authorized to delete table bucket policy") h.writeError(w, http.StatusForbidden, ErrCodeAccessDenied, "not authorized to delete table bucket policy")
return NewAuthError("DeleteTableBucketPolicy", principal, "not authorized to delete table bucket policy") return NewAuthError("DeleteTableBucketPolicy", principal, "not authorized to delete table bucket policy")
@@ -346,6 +349,7 @@ func (h *S3TablesHandler) handlePutTablePolicy(w http.ResponseWriter, r *http.Re
Namespace: namespaceName, Namespace: namespaceName,
TableName: tableName, TableName: tableName,
IdentityActions: identityActions, IdentityActions: identityActions,
DefaultAllow: h.defaultAllow,
}) { }) {
h.writeError(w, http.StatusForbidden, ErrCodeAccessDenied, "not authorized to put table policy") h.writeError(w, http.StatusForbidden, ErrCodeAccessDenied, "not authorized to put table policy")
return NewAuthError("PutTablePolicy", principal, "not authorized to put table policy") return NewAuthError("PutTablePolicy", principal, "not authorized to put table policy")
@@ -453,6 +457,7 @@ func (h *S3TablesHandler) handleGetTablePolicy(w http.ResponseWriter, r *http.Re
Namespace: namespaceName, Namespace: namespaceName,
TableName: tableName, TableName: tableName,
IdentityActions: identityActions, IdentityActions: identityActions,
DefaultAllow: h.defaultAllow,
}) { }) {
h.writeError(w, http.StatusForbidden, ErrCodeAccessDenied, "not authorized to get table policy") h.writeError(w, http.StatusForbidden, ErrCodeAccessDenied, "not authorized to get table policy")
return NewAuthError("GetTablePolicy", principal, "not authorized to get table policy") return NewAuthError("GetTablePolicy", principal, "not authorized to get table policy")
@@ -542,6 +547,7 @@ func (h *S3TablesHandler) handleDeleteTablePolicy(w http.ResponseWriter, r *http
Namespace: namespaceName, Namespace: namespaceName,
TableName: tableName, TableName: tableName,
IdentityActions: identityActions, IdentityActions: identityActions,
DefaultAllow: h.defaultAllow,
}) { }) {
h.writeError(w, http.StatusForbidden, ErrCodeAccessDenied, "not authorized to delete table policy") h.writeError(w, http.StatusForbidden, ErrCodeAccessDenied, "not authorized to delete table policy")
return NewAuthError("DeleteTablePolicy", principal, "not authorized to delete table policy") return NewAuthError("DeleteTablePolicy", principal, "not authorized to delete table policy")
@@ -640,6 +646,7 @@ func (h *S3TablesHandler) handleTagResource(w http.ResponseWriter, r *http.Reque
TagKeys: requestTagKeys, TagKeys: requestTagKeys,
ResourceTags: existingTags, ResourceTags: existingTags,
IdentityActions: identityActions, IdentityActions: identityActions,
DefaultAllow: h.defaultAllow,
}) { }) {
return NewAuthError("TagResource", principal, "not authorized to tag resource") return NewAuthError("TagResource", principal, "not authorized to tag resource")
} }
@@ -757,6 +764,7 @@ func (h *S3TablesHandler) handleListTagsForResource(w http.ResponseWriter, r *ht
TableBucketTags: bucketTags, TableBucketTags: bucketTags,
ResourceTags: tags, ResourceTags: tags,
IdentityActions: identityActions, IdentityActions: identityActions,
DefaultAllow: h.defaultAllow,
}) { }) {
return NewAuthError("ListTagsForResource", principal, "not authorized to list tags for resource") return NewAuthError("ListTagsForResource", principal, "not authorized to list tags for resource")
} }
@@ -864,6 +872,7 @@ func (h *S3TablesHandler) handleUntagResource(w http.ResponseWriter, r *http.Req
TagKeys: req.TagKeys, TagKeys: req.TagKeys,
ResourceTags: tags, ResourceTags: tags,
IdentityActions: identityActions, IdentityActions: identityActions,
DefaultAllow: h.defaultAllow,
}) { }) {
return NewAuthError("UntagResource", principal, "not authorized to untag resource") return NewAuthError("UntagResource", principal, "not authorized to untag resource")
} }

View File

@@ -145,6 +145,7 @@ func (h *S3TablesHandler) handleCreateTable(w http.ResponseWriter, r *http.Reque
TagKeys: mapKeys(req.Tags), TagKeys: mapKeys(req.Tags),
TableBucketTags: bucketTags, TableBucketTags: bucketTags,
IdentityActions: identityActions, IdentityActions: identityActions,
DefaultAllow: h.defaultAllow,
}) })
bucketAllowed := CheckPermissionWithContext("CreateTable", accountID, bucketMetadata.OwnerAccountID, bucketPolicy, bucketARN, &PolicyContext{ bucketAllowed := CheckPermissionWithContext("CreateTable", accountID, bucketMetadata.OwnerAccountID, bucketPolicy, bucketARN, &PolicyContext{
TableBucketName: bucketName, TableBucketName: bucketName,
@@ -154,6 +155,7 @@ func (h *S3TablesHandler) handleCreateTable(w http.ResponseWriter, r *http.Reque
TagKeys: mapKeys(req.Tags), TagKeys: mapKeys(req.Tags),
TableBucketTags: bucketTags, TableBucketTags: bucketTags,
IdentityActions: identityActions, IdentityActions: identityActions,
DefaultAllow: h.defaultAllow,
}) })
if !nsAllowed && !bucketAllowed { if !nsAllowed && !bucketAllowed {
@@ -390,6 +392,7 @@ func (h *S3TablesHandler) handleGetTable(w http.ResponseWriter, r *http.Request,
TableBucketTags: bucketTags, TableBucketTags: bucketTags,
ResourceTags: tableTags, ResourceTags: tableTags,
IdentityActions: identityActions, IdentityActions: identityActions,
DefaultAllow: h.defaultAllow,
}) })
bucketAllowed := CheckPermissionWithContext("GetTable", accountID, bucketMetadata.OwnerAccountID, bucketPolicy, bucketARN, &PolicyContext{ bucketAllowed := CheckPermissionWithContext("GetTable", accountID, bucketMetadata.OwnerAccountID, bucketPolicy, bucketARN, &PolicyContext{
TableBucketName: bucketName, TableBucketName: bucketName,
@@ -398,6 +401,7 @@ func (h *S3TablesHandler) handleGetTable(w http.ResponseWriter, r *http.Request,
TableBucketTags: bucketTags, TableBucketTags: bucketTags,
ResourceTags: tableTags, ResourceTags: tableTags,
IdentityActions: identityActions, IdentityActions: identityActions,
DefaultAllow: h.defaultAllow,
}) })
if !tableAllowed && !bucketAllowed { if !tableAllowed && !bucketAllowed {
@@ -527,12 +531,14 @@ func (h *S3TablesHandler) handleListTables(w http.ResponseWriter, r *http.Reques
Namespace: namespaceName, Namespace: namespaceName,
TableBucketTags: bucketTags, TableBucketTags: bucketTags,
IdentityActions: identityActions, IdentityActions: identityActions,
DefaultAllow: h.defaultAllow,
}) })
bucketAllowed := CheckPermissionWithContext("ListTables", accountID, bucketMeta.OwnerAccountID, bucketPolicy, bucketARN, &PolicyContext{ bucketAllowed := CheckPermissionWithContext("ListTables", accountID, bucketMeta.OwnerAccountID, bucketPolicy, bucketARN, &PolicyContext{
TableBucketName: bucketName, TableBucketName: bucketName,
Namespace: namespaceName, Namespace: namespaceName,
TableBucketTags: bucketTags, TableBucketTags: bucketTags,
IdentityActions: identityActions, IdentityActions: identityActions,
DefaultAllow: h.defaultAllow,
}) })
if !nsAllowed && !bucketAllowed { if !nsAllowed && !bucketAllowed {
return ErrAccessDenied return ErrAccessDenied
@@ -574,6 +580,7 @@ func (h *S3TablesHandler) handleListTables(w http.ResponseWriter, r *http.Reques
TableBucketName: bucketName, TableBucketName: bucketName,
TableBucketTags: bucketTags, TableBucketTags: bucketTags,
IdentityActions: identityActions, IdentityActions: identityActions,
DefaultAllow: h.defaultAllow,
}) { }) {
return ErrAccessDenied return ErrAccessDenied
} }
@@ -910,6 +917,7 @@ func (h *S3TablesHandler) handleDeleteTable(w http.ResponseWriter, r *http.Reque
TableBucketTags: bucketTags, TableBucketTags: bucketTags,
ResourceTags: tableTags, ResourceTags: tableTags,
IdentityActions: identityActions, IdentityActions: identityActions,
DefaultAllow: h.defaultAllow,
}) })
bucketAllowed := CheckPermissionWithContext("DeleteTable", principal, bucketMetadata.OwnerAccountID, bucketPolicy, bucketARN, &PolicyContext{ bucketAllowed := CheckPermissionWithContext("DeleteTable", principal, bucketMetadata.OwnerAccountID, bucketPolicy, bucketARN, &PolicyContext{
TableBucketName: bucketName, TableBucketName: bucketName,
@@ -918,6 +926,7 @@ func (h *S3TablesHandler) handleDeleteTable(w http.ResponseWriter, r *http.Reque
TableBucketTags: bucketTags, TableBucketTags: bucketTags,
ResourceTags: tableTags, ResourceTags: tableTags,
IdentityActions: identityActions, IdentityActions: identityActions,
DefaultAllow: h.defaultAllow,
}) })
if !tableAllowed && !bucketAllowed { if !tableAllowed && !bucketAllowed {
h.writeError(w, http.StatusForbidden, ErrCodeAccessDenied, "not authorized to delete table") h.writeError(w, http.StatusForbidden, ErrCodeAccessDenied, "not authorized to delete table")
@@ -1053,6 +1062,7 @@ func (h *S3TablesHandler) handleUpdateTable(w http.ResponseWriter, r *http.Reque
TableBucketTags: bucketTags, TableBucketTags: bucketTags,
ResourceTags: tableTags, ResourceTags: tableTags,
IdentityActions: identityActions, IdentityActions: identityActions,
DefaultAllow: h.defaultAllow,
}) })
bucketAllowed := CheckPermissionWithContext("UpdateTable", principal, bucketMetadata.OwnerAccountID, bucketPolicy, bucketARN, &PolicyContext{ bucketAllowed := CheckPermissionWithContext("UpdateTable", principal, bucketMetadata.OwnerAccountID, bucketPolicy, bucketARN, &PolicyContext{
TableBucketName: bucketName, TableBucketName: bucketName,
@@ -1061,6 +1071,7 @@ func (h *S3TablesHandler) handleUpdateTable(w http.ResponseWriter, r *http.Reque
TableBucketTags: bucketTags, TableBucketTags: bucketTags,
ResourceTags: tableTags, ResourceTags: tableTags,
IdentityActions: identityActions, IdentityActions: identityActions,
DefaultAllow: h.defaultAllow,
}) })
if !tableAllowed && !bucketAllowed { if !tableAllowed && !bucketAllowed {

View File

@@ -20,7 +20,10 @@ type Manager struct {
// NewManager creates a new Manager. // NewManager creates a new Manager.
func NewManager() *Manager { func NewManager() *Manager {
return &Manager{handler: NewS3TablesHandler()} m := &Manager{handler: NewS3TablesHandler()}
// Default to allowing access when IAM is not configured
m.handler.SetDefaultAllow(true)
return m
} }
// SetRegion sets the AWS region for ARN generation. // SetRegion sets the AWS region for ARN generation.
@@ -33,6 +36,11 @@ func (m *Manager) SetAccountID(accountID string) {
m.handler.SetAccountID(accountID) m.handler.SetAccountID(accountID)
} }
// SetDefaultAllow sets whether to allow access by default.
func (m *Manager) SetDefaultAllow(allow bool) {
m.handler.SetDefaultAllow(allow)
}
// Execute runs an S3 Tables operation and decodes the response into resp (if provided). // Execute runs an S3 Tables operation and decodes the response into resp (if provided).
func (m *Manager) Execute(ctx context.Context, filerClient FilerClient, operation string, req interface{}, resp interface{}, identity string) error { func (m *Manager) Execute(ctx context.Context, filerClient FilerClient, operation string, req interface{}, resp interface{}, identity string) error {
body, err := json.Marshal(req) body, err := json.Marshal(req)

View File

@@ -86,6 +86,7 @@ type PolicyContext struct {
SSEAlgorithm string SSEAlgorithm string
KMSKeyArn string KMSKeyArn string
StorageClass string StorageClass string
DefaultAllow bool
} }
// CheckPermissionWithResource checks if a principal has permission to perform an operation on a specific resource // CheckPermissionWithResource checks if a principal has permission to perform an operation on a specific resource
@@ -117,17 +118,30 @@ func CheckPermissionWithContext(operation, principal, owner, resourcePolicy, res
} }
func checkPermission(operation, principal, owner, resourcePolicy, resourceARN string, ctx *PolicyContext) bool { func checkPermission(operation, principal, owner, resourcePolicy, resourceARN string, ctx *PolicyContext) bool {
fmt.Printf("DEBUG: checkPermission op=%s princ=%s owner=%s policyLen=%d defaultAllow=%v\n",
operation, principal, owner, len(resourcePolicy), ctx != nil && ctx.DefaultAllow)
if resourcePolicy != "" {
fmt.Printf("DEBUG: policy content: %s\n", resourcePolicy)
}
// Owner always has permission // Owner always has permission
if principal == owner { if principal == owner {
fmt.Printf("DEBUG: Allowed by Owner check\n")
return true return true
} }
if hasIdentityPermission(operation, ctx) { if hasIdentityPermission(operation, ctx) {
fmt.Printf("DEBUG: Allowed by Identity check\n")
return true return true
} }
// If no policy is provided, deny access (default deny) // If no policy is provided, use default allow if enabled
if resourcePolicy == "" { if resourcePolicy == "" {
if ctx != nil && ctx.DefaultAllow {
fmt.Printf("DEBUG: Allowed by DefaultAllow\n")
return true
}
fmt.Printf("DEBUG: Denied by DefaultAllow=false (no policy)\n")
return false return false
} }
@@ -177,7 +191,16 @@ func checkPermission(operation, principal, owner, resourcePolicy, resourceARN st
} }
} }
return hasAllow if hasAllow {
return true
}
// If no statement matched, use default allow if enabled
if ctx != nil && ctx.DefaultAllow {
return true
}
return false
} }
func hasIdentityPermission(operation string, ctx *PolicyContext) bool { func hasIdentityPermission(operation string, ctx *PolicyContext) bool {

View File

@@ -206,3 +206,48 @@ func TestEvaluatePolicyWithConditions(t *testing.T) {
}) })
} }
} }
func TestCheckPermissionWithDefaultAllow(t *testing.T) {
tests := []struct {
name string
defaultAllow bool
policy string
expected bool
}{
{
"default deny (no policy, DefaultAllow=false)",
false,
"",
false,
},
{
"default allow (no policy, DefaultAllow=true)",
true,
"",
true,
},
{
"explicit deny overrides DefaultAllow=true",
true,
`{"Statement":[{"Effect":"Deny","Principal":"*","Action":"s3tables:GetTable"}]}`,
false,
},
{
"explicit allow works with DefaultAllow=false",
false,
`{"Statement":[{"Effect":"Allow","Principal":"*","Action":"s3tables:GetTable"}]}`,
true,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
ctx := &PolicyContext{
DefaultAllow: tt.defaultAllow,
}
result := CheckPermissionWithContext("s3tables:GetTable", "user123", "owner123", tt.policy, "", ctx)
if result != tt.expected {
t.Errorf("CheckPermissionWithContext() = %v, want %v (DefaultAllow=%v, Policy=%s)", result, tt.expected, tt.defaultAllow, tt.policy)
}
})
}
}

View File

@@ -41,6 +41,9 @@ func (m *mockIAMIntegration) ValidateTrustPolicyForPrincipal(ctx context.Context
func (m *mockIAMIntegration) ValidateSessionToken(ctx context.Context, token string) (*sts.SessionInfo, error) { func (m *mockIAMIntegration) ValidateSessionToken(ctx context.Context, token string) (*sts.SessionInfo, error) {
return nil, nil return nil, nil
} }
func (m *mockIAMIntegration) DefaultAllow() bool {
return true
}
func TestSTSAssumeRolePostBody(t *testing.T) { func TestSTSAssumeRolePostBody(t *testing.T) {
// Setup S3ApiServer with IAM enabled // Setup S3ApiServer with IAM enabled