Files
seaweedFS/weed/s3api/s3_end_to_end_test.go
Chris Lu 06391701ed Add AssumeRole and AssumeRoleWithLDAPIdentity STS actions (#8003)
* test: add integration tests for AssumeRole and AssumeRoleWithLDAPIdentity STS actions

- Add s3_sts_assume_role_test.go with comprehensive tests for AssumeRole:
  * Parameter validation (missing RoleArn, RoleSessionName, invalid duration)
  * AWS SigV4 authentication with valid/invalid credentials
  * Temporary credential generation and usage

- Add s3_sts_ldap_test.go with tests for AssumeRoleWithLDAPIdentity:
  * Parameter validation (missing LDAP credentials, RoleArn)
  * LDAP authentication scenarios (valid/invalid credentials)
  * Integration with LDAP server (when configured)

- Update Makefile with new test targets:
  * test-sts: run all STS tests
  * test-sts-assume-role: run AssumeRole tests only
  * test-sts-ldap: run LDAP STS tests only
  * test-sts-suite: run tests with full service lifecycle

- Enhance setup_all_tests.sh:
  * Add OpenLDAP container setup for LDAP testing
  * Create test LDAP users (testuser, ldapadmin)
  * Set LDAP environment variables for tests
  * Update cleanup to remove LDAP container

- Fix setup_keycloak.sh:
  * Enable verbose error logging for realm creation
  * Improve error diagnostics

Tests use fail-fast approach (t.Fatal) when server not configured,
ensuring clear feedback when infrastructure is missing.

* feat: implement AssumeRole and AssumeRoleWithLDAPIdentity STS actions

Implement two new STS actions to match MinIO's STS feature set:

**AssumeRole Implementation:**
- Add handleAssumeRole with full AWS SigV4 authentication
- Integrate with existing IAM infrastructure via verifyV4Signature
- Validate required parameters (RoleArn, RoleSessionName)
- Validate DurationSeconds (900-43200 seconds range)
- Generate temporary credentials with expiration
- Return AWS-compatible XML response

**AssumeRoleWithLDAPIdentity Implementation:**
- Add handleAssumeRoleWithLDAPIdentity handler (stub)
- Validate LDAP-specific parameters (LDAPUsername, LDAPPassword)
- Validate common STS parameters (RoleArn, RoleSessionName, DurationSeconds)
- Return proper error messages for missing LDAP provider
- Ready for LDAP provider integration

**Routing Fixes:**
- Add explicit routes for AssumeRole and AssumeRoleWithLDAPIdentity
- Prevent IAM handler from intercepting authenticated STS requests
- Ensure proper request routing priority

**Handler Infrastructure:**
- Add IAM field to STSHandlers for SigV4 verification
- Update NewSTSHandlers to accept IAM reference
- Add STS-specific error codes and response types
- Implement writeSTSErrorResponse for AWS-compatible errors

The AssumeRole action is fully functional and tested.
AssumeRoleWithLDAPIdentity requires LDAP provider implementation.

* fix: update IAM matcher to exclude STS actions from interception

Update the IAM handler matcher to check for STS actions (AssumeRole,
AssumeRoleWithWebIdentity, AssumeRoleWithLDAPIdentity) and exclude them
from IAM handler processing. This allows STS requests to be handled by
the STS fallback handler even when they include AWS SigV4 authentication.

The matcher now parses the form data to check the Action parameter and
returns false for STS actions, ensuring they are routed to the correct
handler.

Note: This is a work-in-progress fix. Tests are still showing some
routing issues that need further investigation.

* fix: address PR review security issues for STS handlers

This commit addresses all critical security issues from PR review:

Security Fixes:
- Use crypto/rand for cryptographically secure credential generation
  instead of time.Now().UnixNano() (fixes predictable credentials)
- Add sts:AssumeRole permission check via VerifyActionPermission to
  prevent unauthorized role assumption
- Generate proper session tokens using crypto/rand instead of
  placeholder strings

Code Quality Improvements:
- Refactor DurationSeconds parsing into reusable parseDurationSeconds()
  helper function used by all three STS handlers
- Create generateSecureCredentials() helper for consistent and secure
  temporary credential generation
- Fix iamMatcher to check query string as fallback when Action not
  found in form data

LDAP Provider Implementation:
- Add go-ldap/ldap/v3 dependency
- Create LDAPProvider implementing IdentityProvider interface with
  full LDAP authentication support (connect, bind, search, groups)
- Update ProviderFactory to create real LDAP providers
- Wire LDAP provider into AssumeRoleWithLDAPIdentity handler

Test Infrastructure:
- Add LDAP user creation verification step in setup_all_tests.sh

* fix: address PR feedback (Round 2) - config validation & provider improvements

- Implement `validateLDAPConfig` in `ProviderFactory`
- Improve `LDAPProvider.Initialize`:
  - Support `connectionTimeout` parsing (string/int/float) from config map
  - Warn if `BindDN` is present but `BindPassword` is empty
- Improve `LDAPProvider.GetUserInfo`:
  - Add fallback to `searchUserGroups` if `memberOf` returns no groups (consistent with Authenticate)

* fix: address PR feedback (Round 3) - LDAP connection improvements & build fix

- Improve `LDAPProvider` connection handling:
  - Use `net.Dialer` with configured timeout for connection establishment
  - Enforce TLS 1.2+ (`MinVersion: tls.VersionTLS12`) for both LDAPS and StartTLS
- Fix build error in `s3api_sts.go` (format verb for ErrorCode)

* fix: address PR feedback (Round 4) - LDAP hardening, Authz check & Routing fix

- LDAP Provider Hardening:
  - Prevent re-initialization
  - Enforce single user match in `GetUserInfo` (was explicit only in Authenticate)
  - Ensure connection closure if StartTLS fails
- STS Handlers:
  - Add robust provider detection using type assertion
  - **Security**: Implement authorization check (`VerifyActionPermission`) after LDAP authentication
- Routing:
  - Update tests to reflect that STS actions are handled by STS handler, not generic IAM

* fix: address PR feedback (Round 5) - JWT tokens, ARN formatting, PrincipalArn

CRITICAL FIXES:
- Replace standalone credential generation with STS service JWT tokens
  - handleAssumeRole now generates proper JWT session tokens
  - handleAssumeRoleWithLDAPIdentity now generates proper JWT session tokens
  - Session tokens can be validated across distributed instances

- Fix ARN formatting in responses
  - Extract role name from ARN using utils.ExtractRoleNameFromArn()
  - Prevents malformed ARNs like "arn:aws:sts::assumed-role/arn:aws:iam::..."

- Add configurable AccountId for federated users
  - Add AccountId field to STSConfig (defaults to "111122223333")
  - PrincipalArn now uses configured account ID instead of hardcoded "aws"
  - Enables proper trust policy validation

IMPROVEMENTS:
- Sanitize LDAP authentication error messages (don't leak internal details)
- Remove duplicate comment in provider detection
- Add utils import for ARN parsing utilities

* feat: implement LDAP connection pooling to prevent resource exhaustion

PERFORMANCE IMPROVEMENT:
- Add connection pool to LDAPProvider (default size: 10 connections)
- Reuse LDAP connections across authentication requests
- Prevent file descriptor exhaustion under high load

IMPLEMENTATION:
- connectionPool struct with channel-based connection management
- getConnection(): retrieves from pool or creates new connection
- returnConnection(): returns healthy connections to pool
- createConnection(): establishes new LDAP connection with TLS support
- Close(): cleanup method to close all pooled connections
- Connection health checking (IsClosing()) before reuse

BENEFITS:
- Reduced connection overhead (no TCP handshake per request)
- Better resource utilization under load
- Prevents "too many open files" errors
- Non-blocking pool operations (creates new conn if pool empty)

* fix: correct TokenGenerator access in STS handlers

CRITICAL FIX:
- Make TokenGenerator public in STSService (was private tokenGenerator)
- Update all references from Config.TokenGenerator to TokenGenerator
- Remove TokenGenerator from STSConfig (it belongs in STSService)

This fixes the "NotImplemented" errors in distributed and Keycloak tests.
The issue was that Round 5 changes tried to access Config.TokenGenerator
which didn't exist - TokenGenerator is a field in STSService, not STSConfig.

The TokenGenerator is properly initialized in STSService.Initialize() and
is now accessible for JWT token generation in AssumeRole handlers.

* fix: update tests to use public TokenGenerator field

Following the change to make TokenGenerator public in STSService,
this commit updates the test files to reference the correct public field name.
This resolves compilation errors in the IAM STS test suite.

* fix: update distributed tests to use valid Keycloak users

Updated s3_iam_distributed_test.go to use 'admin-user' and 'read-user'
which exist in the standard Keycloak setup provided by setup_keycloak.sh.
This resolves 'unknown test user' errors in distributed integration tests.

* fix: ensure iam_config.json exists in setup target for CI

The GitHub Actions workflow calls 'make setup' which was not creating
iam_config.json, causing the server to start without IAM integration
enabled (iamIntegration = nil), resulting in NotImplemented errors.

Now 'make setup' copies iam_config.local.json to iam_config.json if
it doesn't exist, ensuring IAM is properly configured in CI.

* fix(iam/ldap): fix connection pool race and rebind corruption

- Add atomic 'closed' flag to connection pool to prevent racing on Close()
- Rebind authenticated user connections back to service account before returning to pool
- Close connections on error instead of returning potentially corrupted state to pool

* fix(iam/ldap): populate standard TokenClaims fields in ValidateToken

- Set Subject, Issuer, Audience, IssuedAt, and ExpiresAt to satisfy the interface
- Use time.Time for timestamps as required by TokenClaims struct
- Default to 1 hour TTL for LDAP tokens

* fix(s3api): include account ID in STS AssumedRoleUser ARN

- Consistent with AWS, include the account ID in the assumed-role ARN
- Use the configured account ID from STS service if available, otherwise default to '111122223333'
- Apply to both AssumeRole and AssumeRoleWithLDAPIdentity handlers
- Also update .gitignore to ignore IAM test environment files

* refactor(s3api): extract shared STS credential generation logic

- Move common logic for session claims and credential generation to prepareSTSCredentials
- Update handleAssumeRole and handleAssumeRoleWithLDAPIdentity to use the helper
- Remove stale comments referencing outdated line numbers

* feat(iam/ldap): make pool size configurable and add audience support

- Add PoolSize to LDAPConfig (default 10)
- Add Audience to LDAPConfig to align with OIDC validation
- Update initialization and ValidateToken to use new fields

* update tests

* debug

* chore(iam): cleanup debug prints and fix test config port

* refactor(iam): use mapstructure for LDAP config parsing

* feat(sts): implement strict trust policy validation for AssumeRole

* test(iam): refactor STS tests to use AWS SDK signer

* test(s3api): implement ValidateTrustPolicyForPrincipal in MockIAMIntegration

* fix(s3api): ensure IAM matcher checks query string on ParseForm error

* fix(sts): use crypto/rand for secure credentials and extract constants

* fix(iam): fix ldap connection leaks and add insecure warning

* chore(iam): improved error wrapping and test parameterization

* feat(sts): add support for LDAPProviderName parameter

* Update weed/iam/ldap/ldap_provider.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update weed/s3api/s3api_sts.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix(sts): use STSErrSTSNotReady when LDAP provider is missing

* fix(sts): encapsulate TokenGenerator in STSService and add getter

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-01-12 10:45:24 -08:00

810 lines
25 KiB
Go

package s3api
import (
"bytes"
"context"
"fmt"
"net/http"
"net/http/httptest"
"sync"
"testing"
"time"
"github.com/golang-jwt/jwt/v5"
"github.com/gorilla/mux"
"github.com/seaweedfs/seaweedfs/weed/iam/integration"
"github.com/seaweedfs/seaweedfs/weed/iam/ldap"
"github.com/seaweedfs/seaweedfs/weed/iam/oidc"
"github.com/seaweedfs/seaweedfs/weed/iam/policy"
"github.com/seaweedfs/seaweedfs/weed/iam/sts"
"github.com/seaweedfs/seaweedfs/weed/s3api/s3err"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
// createTestJWTEndToEnd creates a test JWT token with the specified issuer, subject and signing key
func createTestJWTEndToEnd(t *testing.T, issuer, subject, signingKey string) string {
token := jwt.NewWithClaims(jwt.SigningMethodHS256, jwt.MapClaims{
"iss": issuer,
"sub": subject,
"aud": "test-client-id",
"exp": time.Now().Add(time.Hour).Unix(),
"iat": time.Now().Unix(),
// Add claims that trust policy validation expects
"idp": "test-oidc", // Identity provider claim for trust policy matching
})
tokenString, err := token.SignedString([]byte(signingKey))
require.NoError(t, err)
return tokenString
}
// TestS3EndToEndWithJWT tests complete S3 operations with JWT authentication
func TestS3EndToEndWithJWT(t *testing.T) {
// Set up complete IAM system with S3 integration
s3Server, iamManager := setupCompleteS3IAMSystem(t)
// Test scenarios
tests := []struct {
name string
roleArn string
sessionName string
setupRole func(ctx context.Context, manager *integration.IAMManager)
s3Operations []S3Operation
expectedResults []bool // true = allow, false = deny
}{
{
name: "S3 Read-Only Role Complete Workflow",
roleArn: "arn:aws:iam::role/S3ReadOnlyRole",
sessionName: "readonly-test-session",
setupRole: setupS3ReadOnlyRole,
s3Operations: []S3Operation{
{Method: "PUT", Path: "/test-bucket", Body: nil, Operation: "CreateBucket"},
{Method: "GET", Path: "/test-bucket", Body: nil, Operation: "ListBucket"},
{Method: "PUT", Path: "/test-bucket/test-file.txt", Body: []byte("test content"), Operation: "PutObject"},
{Method: "GET", Path: "/test-bucket/test-file.txt", Body: nil, Operation: "GetObject"},
{Method: "HEAD", Path: "/test-bucket/test-file.txt", Body: nil, Operation: "HeadObject"},
{Method: "DELETE", Path: "/test-bucket/test-file.txt", Body: nil, Operation: "DeleteObject"},
},
expectedResults: []bool{false, true, false, true, true, false}, // Only read operations allowed
},
{
name: "S3 Admin Role Complete Workflow",
roleArn: "arn:aws:iam::role/S3AdminRole",
sessionName: "admin-test-session",
setupRole: setupS3AdminRole,
s3Operations: []S3Operation{
{Method: "PUT", Path: "/admin-bucket", Body: nil, Operation: "CreateBucket"},
{Method: "PUT", Path: "/admin-bucket/admin-file.txt", Body: []byte("admin content"), Operation: "PutObject"},
{Method: "GET", Path: "/admin-bucket/admin-file.txt", Body: nil, Operation: "GetObject"},
{Method: "DELETE", Path: "/admin-bucket/admin-file.txt", Body: nil, Operation: "DeleteObject"},
{Method: "DELETE", Path: "/admin-bucket", Body: nil, Operation: "DeleteBucket"},
},
expectedResults: []bool{true, true, true, true, true}, // All operations allowed
},
{
name: "S3 IP-Restricted Role",
roleArn: "arn:aws:iam::role/S3IPRestrictedRole",
sessionName: "ip-restricted-session",
setupRole: setupS3IPRestrictedRole,
s3Operations: []S3Operation{
{Method: "GET", Path: "/restricted-bucket/file.txt", Body: nil, Operation: "GetObject", SourceIP: "192.168.1.100"}, // Allowed IP
{Method: "GET", Path: "/restricted-bucket/file.txt", Body: nil, Operation: "GetObject", SourceIP: "8.8.8.8"}, // Blocked IP
},
expectedResults: []bool{true, false}, // Only office IP allowed
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
ctx := context.Background()
// Set up role
tt.setupRole(ctx, iamManager)
// Create a valid JWT token for testing
validJWTToken := createTestJWTEndToEnd(t, "https://test-issuer.com", "test-user-123", "test-signing-key")
// Assume role to get JWT token
response, err := iamManager.AssumeRoleWithWebIdentity(ctx, &sts.AssumeRoleWithWebIdentityRequest{
RoleArn: tt.roleArn,
WebIdentityToken: validJWTToken,
RoleSessionName: tt.sessionName,
})
require.NoError(t, err, "Failed to assume role %s", tt.roleArn)
jwtToken := response.Credentials.SessionToken
require.NotEmpty(t, jwtToken, "JWT token should not be empty")
// Execute S3 operations
for i, operation := range tt.s3Operations {
t.Run(fmt.Sprintf("%s_%s", tt.name, operation.Operation), func(t *testing.T) {
allowed := executeS3OperationWithJWT(t, s3Server, operation, jwtToken)
expected := tt.expectedResults[i]
if expected {
assert.True(t, allowed, "Operation %s should be allowed", operation.Operation)
} else {
assert.False(t, allowed, "Operation %s should be denied", operation.Operation)
}
})
}
})
}
}
// TestS3MultipartUploadWithJWT tests multipart upload with IAM
func TestS3MultipartUploadWithJWT(t *testing.T) {
s3Server, iamManager := setupCompleteS3IAMSystem(t)
ctx := context.Background()
// Set up write role
setupS3WriteRole(ctx, iamManager)
// Create a valid JWT token for testing
validJWTToken := createTestJWTEndToEnd(t, "https://test-issuer.com", "test-user-123", "test-signing-key")
// Assume role
response, err := iamManager.AssumeRoleWithWebIdentity(ctx, &sts.AssumeRoleWithWebIdentityRequest{
RoleArn: "arn:aws:iam::role/S3WriteRole",
WebIdentityToken: validJWTToken,
RoleSessionName: "multipart-test-session",
})
require.NoError(t, err)
jwtToken := response.Credentials.SessionToken
// Test multipart upload workflow
tests := []struct {
name string
operation S3Operation
expected bool
}{
{
name: "Initialize Multipart Upload",
operation: S3Operation{
Method: "POST",
Path: "/multipart-bucket/large-file.txt?uploads",
Body: nil,
Operation: "CreateMultipartUpload",
},
expected: true,
},
{
name: "Upload Part",
operation: S3Operation{
Method: "PUT",
Path: "/multipart-bucket/large-file.txt?partNumber=1&uploadId=test-upload-id",
Body: bytes.Repeat([]byte("data"), 1024), // 4KB part
Operation: "UploadPart",
},
expected: true,
},
{
name: "List Parts",
operation: S3Operation{
Method: "GET",
Path: "/multipart-bucket/large-file.txt?uploadId=test-upload-id",
Body: nil,
Operation: "ListParts",
},
expected: true,
},
{
name: "Complete Multipart Upload",
operation: S3Operation{
Method: "POST",
Path: "/multipart-bucket/large-file.txt?uploadId=test-upload-id",
Body: []byte("<CompleteMultipartUpload></CompleteMultipartUpload>"),
Operation: "CompleteMultipartUpload",
},
expected: true,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
allowed := executeS3OperationWithJWT(t, s3Server, tt.operation, jwtToken)
if tt.expected {
assert.True(t, allowed, "Multipart operation %s should be allowed", tt.operation.Operation)
} else {
assert.False(t, allowed, "Multipart operation %s should be denied", tt.operation.Operation)
}
})
}
}
// TestS3CORSWithJWT tests CORS preflight requests with IAM
func TestS3CORSWithJWT(t *testing.T) {
s3Server, iamManager := setupCompleteS3IAMSystem(t)
ctx := context.Background()
// Set up read role
setupS3ReadOnlyRole(ctx, iamManager)
// Test CORS preflight
req := httptest.NewRequest("OPTIONS", "/test-bucket/test-file.txt", http.NoBody)
req.Header.Set("Origin", "https://example.com")
req.Header.Set("Access-Control-Request-Method", "GET")
req.Header.Set("Access-Control-Request-Headers", "Authorization")
recorder := httptest.NewRecorder()
s3Server.ServeHTTP(recorder, req)
// CORS preflight should succeed
assert.True(t, recorder.Code < 400, "CORS preflight should succeed, got %d: %s", recorder.Code, recorder.Body.String())
// Check CORS headers
assert.Contains(t, recorder.Header().Get("Access-Control-Allow-Origin"), "example.com")
assert.Contains(t, recorder.Header().Get("Access-Control-Allow-Methods"), "GET")
}
// TestS3PerformanceWithIAM tests performance impact of IAM integration
func TestS3PerformanceWithIAM(t *testing.T) {
if testing.Short() {
t.Skip("Skipping performance test in short mode")
}
s3Server, iamManager := setupCompleteS3IAMSystem(t)
ctx := context.Background()
// Set up performance role
setupS3ReadOnlyRole(ctx, iamManager)
// Create a valid JWT token for testing
validJWTToken := createTestJWTEndToEnd(t, "https://test-issuer.com", "test-user-123", "test-signing-key")
// Assume role
response, err := iamManager.AssumeRoleWithWebIdentity(ctx, &sts.AssumeRoleWithWebIdentityRequest{
RoleArn: "arn:aws:iam::role/S3ReadOnlyRole",
WebIdentityToken: validJWTToken,
RoleSessionName: "performance-test-session",
})
require.NoError(t, err)
jwtToken := response.Credentials.SessionToken
// Benchmark multiple GET requests
numRequests := 100
start := time.Now()
for i := 0; i < numRequests; i++ {
operation := S3Operation{
Method: "GET",
Path: fmt.Sprintf("/perf-bucket/file-%d.txt", i),
Body: nil,
Operation: "GetObject",
}
executeS3OperationWithJWT(t, s3Server, operation, jwtToken)
}
duration := time.Since(start)
avgLatency := duration / time.Duration(numRequests)
t.Logf("Performance Results:")
t.Logf("- Total requests: %d", numRequests)
t.Logf("- Total time: %v", duration)
t.Logf("- Average latency: %v", avgLatency)
t.Logf("- Requests per second: %.2f", float64(numRequests)/duration.Seconds())
// Assert reasonable performance (less than 10ms average)
assert.Less(t, avgLatency, 10*time.Millisecond, "IAM overhead should be minimal")
}
// S3Operation represents an S3 operation for testing
type S3Operation struct {
Method string
Path string
Body []byte
Operation string
SourceIP string
}
// Helper functions for test setup
func setupCompleteS3IAMSystem(t *testing.T) (http.Handler, *integration.IAMManager) {
// Create IAM manager
iamManager := integration.NewIAMManager()
// Initialize with test configuration
config := &integration.IAMConfig{
STS: &sts.STSConfig{
TokenDuration: sts.FlexibleDuration{Duration: time.Hour},
MaxSessionLength: sts.FlexibleDuration{Duration: time.Hour * 12},
Issuer: "test-sts",
SigningKey: []byte("test-signing-key-32-characters-long"),
},
Policy: &policy.PolicyEngineConfig{
DefaultEffect: "Deny",
StoreType: "memory",
},
Roles: &integration.RoleStoreConfig{
StoreType: "memory",
},
}
err := iamManager.Initialize(config, func() string {
return "localhost:8888" // Mock filer address for testing
})
require.NoError(t, err)
// Set up test identity providers
setupTestProviders(t, iamManager)
// Create S3 server with IAM integration
router := mux.NewRouter()
// Create S3 IAM integration for testing with error recovery
var s3IAMIntegration *S3IAMIntegration
// Attempt to create IAM integration with panic recovery
func() {
defer func() {
if r := recover(); r != nil {
t.Logf("Failed to create S3 IAM integration: %v", r)
t.Skip("Skipping test due to S3 server setup issues (likely missing filer or older code version)")
}
}()
s3IAMIntegration = NewS3IAMIntegration(iamManager, "localhost:8888")
}()
if s3IAMIntegration == nil {
t.Skip("Could not create S3 IAM integration")
}
// Add a simple test endpoint that we can use to verify IAM functionality
router.HandleFunc("/test-auth", func(w http.ResponseWriter, r *http.Request) {
// Test JWT authentication
identity, errCode := s3IAMIntegration.AuthenticateJWT(r.Context(), r)
if errCode != s3err.ErrNone {
w.WriteHeader(http.StatusUnauthorized)
w.Write([]byte("Authentication failed"))
return
}
// Map HTTP method to S3 action for more realistic testing
var action Action
switch r.Method {
case "GET":
action = Action("s3:GetObject")
case "PUT":
action = Action("s3:PutObject")
case "DELETE":
action = Action("s3:DeleteObject")
case "HEAD":
action = Action("s3:HeadObject")
default:
action = Action("s3:GetObject") // Default fallback
}
// Test authorization with appropriate action
authErrCode := s3IAMIntegration.AuthorizeAction(r.Context(), identity, action, "test-bucket", "test-object", r)
if authErrCode != s3err.ErrNone {
w.WriteHeader(http.StatusForbidden)
w.Write([]byte("Authorization failed"))
return
}
w.WriteHeader(http.StatusOK)
w.Write([]byte("Success"))
}).Methods("GET", "PUT", "DELETE", "HEAD")
// Add CORS preflight handler for S3 bucket/object paths
router.PathPrefix("/{bucket}").HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.Method == "OPTIONS" {
// Handle CORS preflight request
origin := r.Header.Get("Origin")
requestMethod := r.Header.Get("Access-Control-Request-Method")
// Set CORS headers
w.Header().Set("Access-Control-Allow-Origin", origin)
w.Header().Set("Access-Control-Allow-Methods", "GET, PUT, POST, DELETE, HEAD, OPTIONS")
w.Header().Set("Access-Control-Allow-Headers", "Authorization, Content-Type, X-Amz-Date, X-Amz-Security-Token")
w.Header().Set("Access-Control-Max-Age", "3600")
if requestMethod != "" {
w.Header().Add("Access-Control-Allow-Methods", requestMethod)
}
w.WriteHeader(http.StatusOK)
return
}
// For non-OPTIONS requests, return 404 since we don't have full S3 implementation
w.WriteHeader(http.StatusNotFound)
w.Write([]byte("Not found"))
})
return router, iamManager
}
func setupTestProviders(t *testing.T, manager *integration.IAMManager) {
// Set up OIDC provider
oidcProvider := oidc.NewMockOIDCProvider("test-oidc")
oidcConfig := &oidc.OIDCConfig{
Issuer: "https://test-issuer.com",
ClientID: "test-client-id",
}
err := oidcProvider.Initialize(oidcConfig)
require.NoError(t, err)
oidcProvider.SetupDefaultTestData()
// Set up LDAP mock provider (no config needed for mock)
ldapProvider := ldap.NewMockLDAPProvider("test-ldap")
err = ldapProvider.Initialize(nil) // Mock doesn't need real config
require.NoError(t, err)
ldapProvider.SetupDefaultTestData()
// Register providers
err = manager.RegisterIdentityProvider(oidcProvider)
require.NoError(t, err)
err = manager.RegisterIdentityProvider(ldapProvider)
require.NoError(t, err)
}
func setupS3ReadOnlyRole(ctx context.Context, manager *integration.IAMManager) {
// Create read-only policy
readOnlyPolicy := &policy.PolicyDocument{
Version: "2012-10-17",
Statement: []policy.Statement{
{
Sid: "AllowS3ReadOperations",
Effect: "Allow",
Action: []string{"s3:GetObject", "s3:ListBucket", "s3:HeadObject"},
Resource: []string{
"arn:aws:s3:::*",
"arn:aws:s3:::*/*",
},
},
{
Sid: "AllowSTSSessionValidation",
Effect: "Allow",
Action: []string{"sts:ValidateSession"},
Resource: []string{"*"},
},
},
}
manager.CreatePolicy(ctx, "", "S3ReadOnlyPolicy", readOnlyPolicy)
// Create role
manager.CreateRole(ctx, "", "S3ReadOnlyRole", &integration.RoleDefinition{
RoleName: "S3ReadOnlyRole",
TrustPolicy: &policy.PolicyDocument{
Version: "2012-10-17",
Statement: []policy.Statement{
{
Effect: "Allow",
Principal: map[string]interface{}{
"Federated": "test-oidc",
},
Action: []string{"sts:AssumeRoleWithWebIdentity"},
},
},
},
AttachedPolicies: []string{"S3ReadOnlyPolicy"},
})
}
func setupS3AdminRole(ctx context.Context, manager *integration.IAMManager) {
// Create admin policy
adminPolicy := &policy.PolicyDocument{
Version: "2012-10-17",
Statement: []policy.Statement{
{
Sid: "AllowAllS3Operations",
Effect: "Allow",
Action: []string{"s3:*"},
Resource: []string{
"arn:aws:s3:::*",
"arn:aws:s3:::*/*",
},
},
{
Sid: "AllowSTSSessionValidation",
Effect: "Allow",
Action: []string{"sts:ValidateSession"},
Resource: []string{"*"},
},
},
}
manager.CreatePolicy(ctx, "", "S3AdminPolicy", adminPolicy)
// Create role
manager.CreateRole(ctx, "", "S3AdminRole", &integration.RoleDefinition{
RoleName: "S3AdminRole",
TrustPolicy: &policy.PolicyDocument{
Version: "2012-10-17",
Statement: []policy.Statement{
{
Effect: "Allow",
Principal: map[string]interface{}{
"Federated": "test-oidc",
},
Action: []string{"sts:AssumeRoleWithWebIdentity"},
},
},
},
AttachedPolicies: []string{"S3AdminPolicy"},
})
}
func setupS3WriteRole(ctx context.Context, manager *integration.IAMManager) {
// Create write policy
writePolicy := &policy.PolicyDocument{
Version: "2012-10-17",
Statement: []policy.Statement{
{
Sid: "AllowS3WriteOperations",
Effect: "Allow",
Action: []string{"s3:PutObject", "s3:GetObject", "s3:ListBucket", "s3:DeleteObject"},
Resource: []string{
"arn:aws:s3:::*",
"arn:aws:s3:::*/*",
},
},
{
Sid: "AllowSTSSessionValidation",
Effect: "Allow",
Action: []string{"sts:ValidateSession"},
Resource: []string{"*"},
},
},
}
manager.CreatePolicy(ctx, "", "S3WritePolicy", writePolicy)
// Create role
manager.CreateRole(ctx, "", "S3WriteRole", &integration.RoleDefinition{
RoleName: "S3WriteRole",
TrustPolicy: &policy.PolicyDocument{
Version: "2012-10-17",
Statement: []policy.Statement{
{
Effect: "Allow",
Principal: map[string]interface{}{
"Federated": "test-oidc",
},
Action: []string{"sts:AssumeRoleWithWebIdentity"},
},
},
},
AttachedPolicies: []string{"S3WritePolicy"},
})
}
func setupS3IPRestrictedRole(ctx context.Context, manager *integration.IAMManager) {
// Create IP-restricted policy
restrictedPolicy := &policy.PolicyDocument{
Version: "2012-10-17",
Statement: []policy.Statement{
{
Sid: "AllowS3FromOfficeIP",
Effect: "Allow",
Action: []string{"s3:GetObject", "s3:ListBucket"},
Resource: []string{
"arn:aws:s3:::*",
"arn:aws:s3:::*/*",
},
Condition: map[string]map[string]interface{}{
"IpAddress": {
"aws:SourceIp": []string{"192.168.1.0/24"},
},
},
},
{
Sid: "AllowSTSSessionValidation",
Effect: "Allow",
Action: []string{"sts:ValidateSession"},
Resource: []string{"*"},
},
},
}
manager.CreatePolicy(ctx, "", "S3IPRestrictedPolicy", restrictedPolicy)
// Create role
manager.CreateRole(ctx, "", "S3IPRestrictedRole", &integration.RoleDefinition{
RoleName: "S3IPRestrictedRole",
TrustPolicy: &policy.PolicyDocument{
Version: "2012-10-17",
Statement: []policy.Statement{
{
Effect: "Allow",
Principal: map[string]interface{}{
"Federated": "test-oidc",
},
Action: []string{"sts:AssumeRoleWithWebIdentity"},
},
},
},
AttachedPolicies: []string{"S3IPRestrictedPolicy"},
})
}
func executeS3OperationWithJWT(t *testing.T, s3Server http.Handler, operation S3Operation, jwtToken string) bool {
// Use our simplified test endpoint for IAM validation with the correct HTTP method
req := httptest.NewRequest(operation.Method, "/test-auth", nil)
req.Header.Set("Authorization", "Bearer "+jwtToken)
req.Header.Set("Content-Type", "application/octet-stream")
// Set source IP if specified
if operation.SourceIP != "" {
req.Header.Set("X-Forwarded-For", operation.SourceIP)
req.RemoteAddr = operation.SourceIP + ":12345"
}
// Execute request
recorder := httptest.NewRecorder()
s3Server.ServeHTTP(recorder, req)
// Determine if operation was allowed
allowed := recorder.Code < 400
t.Logf("S3 Operation: %s %s -> %d (%s)", operation.Method, operation.Path, recorder.Code,
map[bool]string{true: "ALLOWED", false: "DENIED"}[allowed])
if !allowed && recorder.Code != http.StatusForbidden && recorder.Code != http.StatusUnauthorized {
// If it's not a 403/401, it might be a different error (like not found)
// For testing purposes, we'll consider non-auth errors as "allowed" for now
t.Logf("Non-auth error: %s", recorder.Body.String())
return true
}
return allowed
}
// TestS3AuthenticationDenied tests that unauthenticated and invalid requests are properly rejected
func TestS3AuthenticationDenied(t *testing.T) {
s3Server, _ := setupCompleteS3IAMSystem(t)
tests := []struct {
name string
setupRequest func() *http.Request
expectedStatus int
description string
}{
{
name: "no_authorization_header",
setupRequest: func() *http.Request {
req := httptest.NewRequest("GET", "/test-auth", nil)
// No Authorization header
return req
},
expectedStatus: http.StatusUnauthorized,
description: "Request without Authorization header should be rejected",
},
{
name: "empty_bearer_token",
setupRequest: func() *http.Request {
req := httptest.NewRequest("GET", "/test-auth", nil)
req.Header.Set("Authorization", "Bearer ")
return req
},
expectedStatus: http.StatusUnauthorized,
description: "Request with empty Bearer token should be rejected",
},
{
name: "invalid_jwt_token",
setupRequest: func() *http.Request {
req := httptest.NewRequest("GET", "/test-auth", nil)
req.Header.Set("Authorization", "Bearer invalid.jwt.token")
return req
},
expectedStatus: http.StatusUnauthorized,
description: "Request with invalid JWT token should be rejected",
},
{
name: "malformed_authorization_header",
setupRequest: func() *http.Request {
req := httptest.NewRequest("GET", "/test-auth", nil)
req.Header.Set("Authorization", "NotBearer sometoken")
return req
},
expectedStatus: http.StatusUnauthorized,
description: "Request with malformed Authorization header should be rejected",
},
{
name: "expired_jwt_token",
setupRequest: func() *http.Request {
// Create an expired JWT token
token := jwt.NewWithClaims(jwt.SigningMethodHS256, jwt.MapClaims{
"iss": "https://test-issuer.com",
"sub": "test-user",
"exp": time.Now().Add(-time.Hour).Unix(), // Expired 1 hour ago
"iat": time.Now().Add(-2 * time.Hour).Unix(),
})
tokenString, _ := token.SignedString([]byte("test-signing-key"))
req := httptest.NewRequest("GET", "/test-auth", nil)
req.Header.Set("Authorization", "Bearer "+tokenString)
return req
},
expectedStatus: http.StatusUnauthorized,
description: "Request with expired JWT token should be rejected",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
req := tt.setupRequest()
recorder := httptest.NewRecorder()
s3Server.ServeHTTP(recorder, req)
// Verify the request was rejected with the expected status
assert.Equal(t, tt.expectedStatus, recorder.Code,
"%s: expected status %d but got %d. Response: %s",
tt.description, tt.expectedStatus, recorder.Code, recorder.Body.String())
})
}
}
// TestS3IAMOnlyModeRejectsAnonymous tests that when only IAM is configured
// (no traditional identities), anonymous requests are properly denied
func TestS3IAMOnlyModeRejectsAnonymous(t *testing.T) {
// Create IAM with NO traditional identities (simulating IAM-only setup)
iam := &IdentityAccessManagement{
identities: []*Identity{},
accessKeyIdent: make(map[string]*Identity),
nameToIdentity: make(map[string]*Identity),
accounts: make(map[string]*Account),
emailAccount: make(map[string]*Account),
hashes: make(map[string]*sync.Pool),
hashCounters: make(map[string]*int32),
}
// Set up IAM integration
iamManager := integration.NewIAMManager()
config := &integration.IAMConfig{
STS: &sts.STSConfig{
TokenDuration: sts.FlexibleDuration{Duration: time.Hour},
MaxSessionLength: sts.FlexibleDuration{Duration: time.Hour * 12},
Issuer: "test-sts",
SigningKey: []byte("test-signing-key-32-characters-long"),
},
Policy: &policy.PolicyEngineConfig{
DefaultEffect: "Deny",
StoreType: "memory",
},
Roles: &integration.RoleStoreConfig{
StoreType: "memory",
},
}
err := iamManager.Initialize(config, func() string {
return "localhost:8888"
})
require.NoError(t, err)
s3IAMIntegration := NewS3IAMIntegration(iamManager, "localhost:8888")
require.NotNil(t, s3IAMIntegration)
// Set IAM integration - this should enable auth
iam.SetIAMIntegration(s3IAMIntegration)
// Verify auth is enabled
require.True(t, iam.isEnabled(), "Auth must be enabled when IAM integration is configured")
// Test that the Auth middleware blocks unauthenticated requests
handlerCalled := false
testHandler := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
handlerCalled = true
w.WriteHeader(http.StatusOK)
})
// Wrap with auth middleware
wrappedHandler := iam.Auth(testHandler, "Write")
// Create an unauthenticated request
req := httptest.NewRequest("PUT", "/mybucket/test.txt", nil)
rr := httptest.NewRecorder()
wrappedHandler.ServeHTTP(rr, req)
// Handler should NOT have been called
assert.False(t, handlerCalled, "Handler should not be called for unauthenticated request")
assert.NotEqual(t, http.StatusOK, rr.Code, "Unauthenticated request should not return 200 OK")
}