* test: add integration tests for AssumeRole and AssumeRoleWithLDAPIdentity STS actions - Add s3_sts_assume_role_test.go with comprehensive tests for AssumeRole: * Parameter validation (missing RoleArn, RoleSessionName, invalid duration) * AWS SigV4 authentication with valid/invalid credentials * Temporary credential generation and usage - Add s3_sts_ldap_test.go with tests for AssumeRoleWithLDAPIdentity: * Parameter validation (missing LDAP credentials, RoleArn) * LDAP authentication scenarios (valid/invalid credentials) * Integration with LDAP server (when configured) - Update Makefile with new test targets: * test-sts: run all STS tests * test-sts-assume-role: run AssumeRole tests only * test-sts-ldap: run LDAP STS tests only * test-sts-suite: run tests with full service lifecycle - Enhance setup_all_tests.sh: * Add OpenLDAP container setup for LDAP testing * Create test LDAP users (testuser, ldapadmin) * Set LDAP environment variables for tests * Update cleanup to remove LDAP container - Fix setup_keycloak.sh: * Enable verbose error logging for realm creation * Improve error diagnostics Tests use fail-fast approach (t.Fatal) when server not configured, ensuring clear feedback when infrastructure is missing. * feat: implement AssumeRole and AssumeRoleWithLDAPIdentity STS actions Implement two new STS actions to match MinIO's STS feature set: **AssumeRole Implementation:** - Add handleAssumeRole with full AWS SigV4 authentication - Integrate with existing IAM infrastructure via verifyV4Signature - Validate required parameters (RoleArn, RoleSessionName) - Validate DurationSeconds (900-43200 seconds range) - Generate temporary credentials with expiration - Return AWS-compatible XML response **AssumeRoleWithLDAPIdentity Implementation:** - Add handleAssumeRoleWithLDAPIdentity handler (stub) - Validate LDAP-specific parameters (LDAPUsername, LDAPPassword) - Validate common STS parameters (RoleArn, RoleSessionName, DurationSeconds) - Return proper error messages for missing LDAP provider - Ready for LDAP provider integration **Routing Fixes:** - Add explicit routes for AssumeRole and AssumeRoleWithLDAPIdentity - Prevent IAM handler from intercepting authenticated STS requests - Ensure proper request routing priority **Handler Infrastructure:** - Add IAM field to STSHandlers for SigV4 verification - Update NewSTSHandlers to accept IAM reference - Add STS-specific error codes and response types - Implement writeSTSErrorResponse for AWS-compatible errors The AssumeRole action is fully functional and tested. AssumeRoleWithLDAPIdentity requires LDAP provider implementation. * fix: update IAM matcher to exclude STS actions from interception Update the IAM handler matcher to check for STS actions (AssumeRole, AssumeRoleWithWebIdentity, AssumeRoleWithLDAPIdentity) and exclude them from IAM handler processing. This allows STS requests to be handled by the STS fallback handler even when they include AWS SigV4 authentication. The matcher now parses the form data to check the Action parameter and returns false for STS actions, ensuring they are routed to the correct handler. Note: This is a work-in-progress fix. Tests are still showing some routing issues that need further investigation. * fix: address PR review security issues for STS handlers This commit addresses all critical security issues from PR review: Security Fixes: - Use crypto/rand for cryptographically secure credential generation instead of time.Now().UnixNano() (fixes predictable credentials) - Add sts:AssumeRole permission check via VerifyActionPermission to prevent unauthorized role assumption - Generate proper session tokens using crypto/rand instead of placeholder strings Code Quality Improvements: - Refactor DurationSeconds parsing into reusable parseDurationSeconds() helper function used by all three STS handlers - Create generateSecureCredentials() helper for consistent and secure temporary credential generation - Fix iamMatcher to check query string as fallback when Action not found in form data LDAP Provider Implementation: - Add go-ldap/ldap/v3 dependency - Create LDAPProvider implementing IdentityProvider interface with full LDAP authentication support (connect, bind, search, groups) - Update ProviderFactory to create real LDAP providers - Wire LDAP provider into AssumeRoleWithLDAPIdentity handler Test Infrastructure: - Add LDAP user creation verification step in setup_all_tests.sh * fix: address PR feedback (Round 2) - config validation & provider improvements - Implement `validateLDAPConfig` in `ProviderFactory` - Improve `LDAPProvider.Initialize`: - Support `connectionTimeout` parsing (string/int/float) from config map - Warn if `BindDN` is present but `BindPassword` is empty - Improve `LDAPProvider.GetUserInfo`: - Add fallback to `searchUserGroups` if `memberOf` returns no groups (consistent with Authenticate) * fix: address PR feedback (Round 3) - LDAP connection improvements & build fix - Improve `LDAPProvider` connection handling: - Use `net.Dialer` with configured timeout for connection establishment - Enforce TLS 1.2+ (`MinVersion: tls.VersionTLS12`) for both LDAPS and StartTLS - Fix build error in `s3api_sts.go` (format verb for ErrorCode) * fix: address PR feedback (Round 4) - LDAP hardening, Authz check & Routing fix - LDAP Provider Hardening: - Prevent re-initialization - Enforce single user match in `GetUserInfo` (was explicit only in Authenticate) - Ensure connection closure if StartTLS fails - STS Handlers: - Add robust provider detection using type assertion - **Security**: Implement authorization check (`VerifyActionPermission`) after LDAP authentication - Routing: - Update tests to reflect that STS actions are handled by STS handler, not generic IAM * fix: address PR feedback (Round 5) - JWT tokens, ARN formatting, PrincipalArn CRITICAL FIXES: - Replace standalone credential generation with STS service JWT tokens - handleAssumeRole now generates proper JWT session tokens - handleAssumeRoleWithLDAPIdentity now generates proper JWT session tokens - Session tokens can be validated across distributed instances - Fix ARN formatting in responses - Extract role name from ARN using utils.ExtractRoleNameFromArn() - Prevents malformed ARNs like "arn:aws:sts::assumed-role/arn:aws:iam::..." - Add configurable AccountId for federated users - Add AccountId field to STSConfig (defaults to "111122223333") - PrincipalArn now uses configured account ID instead of hardcoded "aws" - Enables proper trust policy validation IMPROVEMENTS: - Sanitize LDAP authentication error messages (don't leak internal details) - Remove duplicate comment in provider detection - Add utils import for ARN parsing utilities * feat: implement LDAP connection pooling to prevent resource exhaustion PERFORMANCE IMPROVEMENT: - Add connection pool to LDAPProvider (default size: 10 connections) - Reuse LDAP connections across authentication requests - Prevent file descriptor exhaustion under high load IMPLEMENTATION: - connectionPool struct with channel-based connection management - getConnection(): retrieves from pool or creates new connection - returnConnection(): returns healthy connections to pool - createConnection(): establishes new LDAP connection with TLS support - Close(): cleanup method to close all pooled connections - Connection health checking (IsClosing()) before reuse BENEFITS: - Reduced connection overhead (no TCP handshake per request) - Better resource utilization under load - Prevents "too many open files" errors - Non-blocking pool operations (creates new conn if pool empty) * fix: correct TokenGenerator access in STS handlers CRITICAL FIX: - Make TokenGenerator public in STSService (was private tokenGenerator) - Update all references from Config.TokenGenerator to TokenGenerator - Remove TokenGenerator from STSConfig (it belongs in STSService) This fixes the "NotImplemented" errors in distributed and Keycloak tests. The issue was that Round 5 changes tried to access Config.TokenGenerator which didn't exist - TokenGenerator is a field in STSService, not STSConfig. The TokenGenerator is properly initialized in STSService.Initialize() and is now accessible for JWT token generation in AssumeRole handlers. * fix: update tests to use public TokenGenerator field Following the change to make TokenGenerator public in STSService, this commit updates the test files to reference the correct public field name. This resolves compilation errors in the IAM STS test suite. * fix: update distributed tests to use valid Keycloak users Updated s3_iam_distributed_test.go to use 'admin-user' and 'read-user' which exist in the standard Keycloak setup provided by setup_keycloak.sh. This resolves 'unknown test user' errors in distributed integration tests. * fix: ensure iam_config.json exists in setup target for CI The GitHub Actions workflow calls 'make setup' which was not creating iam_config.json, causing the server to start without IAM integration enabled (iamIntegration = nil), resulting in NotImplemented errors. Now 'make setup' copies iam_config.local.json to iam_config.json if it doesn't exist, ensuring IAM is properly configured in CI. * fix(iam/ldap): fix connection pool race and rebind corruption - Add atomic 'closed' flag to connection pool to prevent racing on Close() - Rebind authenticated user connections back to service account before returning to pool - Close connections on error instead of returning potentially corrupted state to pool * fix(iam/ldap): populate standard TokenClaims fields in ValidateToken - Set Subject, Issuer, Audience, IssuedAt, and ExpiresAt to satisfy the interface - Use time.Time for timestamps as required by TokenClaims struct - Default to 1 hour TTL for LDAP tokens * fix(s3api): include account ID in STS AssumedRoleUser ARN - Consistent with AWS, include the account ID in the assumed-role ARN - Use the configured account ID from STS service if available, otherwise default to '111122223333' - Apply to both AssumeRole and AssumeRoleWithLDAPIdentity handlers - Also update .gitignore to ignore IAM test environment files * refactor(s3api): extract shared STS credential generation logic - Move common logic for session claims and credential generation to prepareSTSCredentials - Update handleAssumeRole and handleAssumeRoleWithLDAPIdentity to use the helper - Remove stale comments referencing outdated line numbers * feat(iam/ldap): make pool size configurable and add audience support - Add PoolSize to LDAPConfig (default 10) - Add Audience to LDAPConfig to align with OIDC validation - Update initialization and ValidateToken to use new fields * update tests * debug * chore(iam): cleanup debug prints and fix test config port * refactor(iam): use mapstructure for LDAP config parsing * feat(sts): implement strict trust policy validation for AssumeRole * test(iam): refactor STS tests to use AWS SDK signer * test(s3api): implement ValidateTrustPolicyForPrincipal in MockIAMIntegration * fix(s3api): ensure IAM matcher checks query string on ParseForm error * fix(sts): use crypto/rand for secure credentials and extract constants * fix(iam): fix ldap connection leaks and add insecure warning * chore(iam): improved error wrapping and test parameterization * feat(sts): add support for LDAPProviderName parameter * Update weed/iam/ldap/ldap_provider.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update weed/s3api/s3api_sts.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * fix(sts): use STSErrSTSNotReady when LDAP provider is missing * fix(sts): encapsulate TokenGenerator in STSService and add getter --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
810 lines
25 KiB
Go
810 lines
25 KiB
Go
package s3api
|
|
|
|
import (
|
|
"bytes"
|
|
"context"
|
|
"fmt"
|
|
"net/http"
|
|
"net/http/httptest"
|
|
"sync"
|
|
"testing"
|
|
"time"
|
|
|
|
"github.com/golang-jwt/jwt/v5"
|
|
"github.com/gorilla/mux"
|
|
"github.com/seaweedfs/seaweedfs/weed/iam/integration"
|
|
"github.com/seaweedfs/seaweedfs/weed/iam/ldap"
|
|
"github.com/seaweedfs/seaweedfs/weed/iam/oidc"
|
|
"github.com/seaweedfs/seaweedfs/weed/iam/policy"
|
|
"github.com/seaweedfs/seaweedfs/weed/iam/sts"
|
|
"github.com/seaweedfs/seaweedfs/weed/s3api/s3err"
|
|
"github.com/stretchr/testify/assert"
|
|
"github.com/stretchr/testify/require"
|
|
)
|
|
|
|
// createTestJWTEndToEnd creates a test JWT token with the specified issuer, subject and signing key
|
|
func createTestJWTEndToEnd(t *testing.T, issuer, subject, signingKey string) string {
|
|
token := jwt.NewWithClaims(jwt.SigningMethodHS256, jwt.MapClaims{
|
|
"iss": issuer,
|
|
"sub": subject,
|
|
"aud": "test-client-id",
|
|
"exp": time.Now().Add(time.Hour).Unix(),
|
|
"iat": time.Now().Unix(),
|
|
// Add claims that trust policy validation expects
|
|
"idp": "test-oidc", // Identity provider claim for trust policy matching
|
|
})
|
|
|
|
tokenString, err := token.SignedString([]byte(signingKey))
|
|
require.NoError(t, err)
|
|
return tokenString
|
|
}
|
|
|
|
// TestS3EndToEndWithJWT tests complete S3 operations with JWT authentication
|
|
func TestS3EndToEndWithJWT(t *testing.T) {
|
|
// Set up complete IAM system with S3 integration
|
|
s3Server, iamManager := setupCompleteS3IAMSystem(t)
|
|
|
|
// Test scenarios
|
|
tests := []struct {
|
|
name string
|
|
roleArn string
|
|
sessionName string
|
|
setupRole func(ctx context.Context, manager *integration.IAMManager)
|
|
s3Operations []S3Operation
|
|
expectedResults []bool // true = allow, false = deny
|
|
}{
|
|
{
|
|
name: "S3 Read-Only Role Complete Workflow",
|
|
roleArn: "arn:aws:iam::role/S3ReadOnlyRole",
|
|
sessionName: "readonly-test-session",
|
|
setupRole: setupS3ReadOnlyRole,
|
|
s3Operations: []S3Operation{
|
|
{Method: "PUT", Path: "/test-bucket", Body: nil, Operation: "CreateBucket"},
|
|
{Method: "GET", Path: "/test-bucket", Body: nil, Operation: "ListBucket"},
|
|
{Method: "PUT", Path: "/test-bucket/test-file.txt", Body: []byte("test content"), Operation: "PutObject"},
|
|
{Method: "GET", Path: "/test-bucket/test-file.txt", Body: nil, Operation: "GetObject"},
|
|
{Method: "HEAD", Path: "/test-bucket/test-file.txt", Body: nil, Operation: "HeadObject"},
|
|
{Method: "DELETE", Path: "/test-bucket/test-file.txt", Body: nil, Operation: "DeleteObject"},
|
|
},
|
|
expectedResults: []bool{false, true, false, true, true, false}, // Only read operations allowed
|
|
},
|
|
{
|
|
name: "S3 Admin Role Complete Workflow",
|
|
roleArn: "arn:aws:iam::role/S3AdminRole",
|
|
sessionName: "admin-test-session",
|
|
setupRole: setupS3AdminRole,
|
|
s3Operations: []S3Operation{
|
|
{Method: "PUT", Path: "/admin-bucket", Body: nil, Operation: "CreateBucket"},
|
|
{Method: "PUT", Path: "/admin-bucket/admin-file.txt", Body: []byte("admin content"), Operation: "PutObject"},
|
|
{Method: "GET", Path: "/admin-bucket/admin-file.txt", Body: nil, Operation: "GetObject"},
|
|
{Method: "DELETE", Path: "/admin-bucket/admin-file.txt", Body: nil, Operation: "DeleteObject"},
|
|
{Method: "DELETE", Path: "/admin-bucket", Body: nil, Operation: "DeleteBucket"},
|
|
},
|
|
expectedResults: []bool{true, true, true, true, true}, // All operations allowed
|
|
},
|
|
{
|
|
name: "S3 IP-Restricted Role",
|
|
roleArn: "arn:aws:iam::role/S3IPRestrictedRole",
|
|
sessionName: "ip-restricted-session",
|
|
setupRole: setupS3IPRestrictedRole,
|
|
s3Operations: []S3Operation{
|
|
{Method: "GET", Path: "/restricted-bucket/file.txt", Body: nil, Operation: "GetObject", SourceIP: "192.168.1.100"}, // Allowed IP
|
|
{Method: "GET", Path: "/restricted-bucket/file.txt", Body: nil, Operation: "GetObject", SourceIP: "8.8.8.8"}, // Blocked IP
|
|
},
|
|
expectedResults: []bool{true, false}, // Only office IP allowed
|
|
},
|
|
}
|
|
|
|
for _, tt := range tests {
|
|
t.Run(tt.name, func(t *testing.T) {
|
|
ctx := context.Background()
|
|
|
|
// Set up role
|
|
tt.setupRole(ctx, iamManager)
|
|
|
|
// Create a valid JWT token for testing
|
|
validJWTToken := createTestJWTEndToEnd(t, "https://test-issuer.com", "test-user-123", "test-signing-key")
|
|
|
|
// Assume role to get JWT token
|
|
response, err := iamManager.AssumeRoleWithWebIdentity(ctx, &sts.AssumeRoleWithWebIdentityRequest{
|
|
RoleArn: tt.roleArn,
|
|
WebIdentityToken: validJWTToken,
|
|
RoleSessionName: tt.sessionName,
|
|
})
|
|
require.NoError(t, err, "Failed to assume role %s", tt.roleArn)
|
|
|
|
jwtToken := response.Credentials.SessionToken
|
|
require.NotEmpty(t, jwtToken, "JWT token should not be empty")
|
|
|
|
// Execute S3 operations
|
|
for i, operation := range tt.s3Operations {
|
|
t.Run(fmt.Sprintf("%s_%s", tt.name, operation.Operation), func(t *testing.T) {
|
|
allowed := executeS3OperationWithJWT(t, s3Server, operation, jwtToken)
|
|
expected := tt.expectedResults[i]
|
|
|
|
if expected {
|
|
assert.True(t, allowed, "Operation %s should be allowed", operation.Operation)
|
|
} else {
|
|
assert.False(t, allowed, "Operation %s should be denied", operation.Operation)
|
|
}
|
|
})
|
|
}
|
|
})
|
|
}
|
|
}
|
|
|
|
// TestS3MultipartUploadWithJWT tests multipart upload with IAM
|
|
func TestS3MultipartUploadWithJWT(t *testing.T) {
|
|
s3Server, iamManager := setupCompleteS3IAMSystem(t)
|
|
ctx := context.Background()
|
|
|
|
// Set up write role
|
|
setupS3WriteRole(ctx, iamManager)
|
|
|
|
// Create a valid JWT token for testing
|
|
validJWTToken := createTestJWTEndToEnd(t, "https://test-issuer.com", "test-user-123", "test-signing-key")
|
|
|
|
// Assume role
|
|
response, err := iamManager.AssumeRoleWithWebIdentity(ctx, &sts.AssumeRoleWithWebIdentityRequest{
|
|
RoleArn: "arn:aws:iam::role/S3WriteRole",
|
|
WebIdentityToken: validJWTToken,
|
|
RoleSessionName: "multipart-test-session",
|
|
})
|
|
require.NoError(t, err)
|
|
|
|
jwtToken := response.Credentials.SessionToken
|
|
|
|
// Test multipart upload workflow
|
|
tests := []struct {
|
|
name string
|
|
operation S3Operation
|
|
expected bool
|
|
}{
|
|
{
|
|
name: "Initialize Multipart Upload",
|
|
operation: S3Operation{
|
|
Method: "POST",
|
|
Path: "/multipart-bucket/large-file.txt?uploads",
|
|
Body: nil,
|
|
Operation: "CreateMultipartUpload",
|
|
},
|
|
expected: true,
|
|
},
|
|
{
|
|
name: "Upload Part",
|
|
operation: S3Operation{
|
|
Method: "PUT",
|
|
Path: "/multipart-bucket/large-file.txt?partNumber=1&uploadId=test-upload-id",
|
|
Body: bytes.Repeat([]byte("data"), 1024), // 4KB part
|
|
Operation: "UploadPart",
|
|
},
|
|
expected: true,
|
|
},
|
|
{
|
|
name: "List Parts",
|
|
operation: S3Operation{
|
|
Method: "GET",
|
|
Path: "/multipart-bucket/large-file.txt?uploadId=test-upload-id",
|
|
Body: nil,
|
|
Operation: "ListParts",
|
|
},
|
|
expected: true,
|
|
},
|
|
{
|
|
name: "Complete Multipart Upload",
|
|
operation: S3Operation{
|
|
Method: "POST",
|
|
Path: "/multipart-bucket/large-file.txt?uploadId=test-upload-id",
|
|
Body: []byte("<CompleteMultipartUpload></CompleteMultipartUpload>"),
|
|
Operation: "CompleteMultipartUpload",
|
|
},
|
|
expected: true,
|
|
},
|
|
}
|
|
|
|
for _, tt := range tests {
|
|
t.Run(tt.name, func(t *testing.T) {
|
|
allowed := executeS3OperationWithJWT(t, s3Server, tt.operation, jwtToken)
|
|
if tt.expected {
|
|
assert.True(t, allowed, "Multipart operation %s should be allowed", tt.operation.Operation)
|
|
} else {
|
|
assert.False(t, allowed, "Multipart operation %s should be denied", tt.operation.Operation)
|
|
}
|
|
})
|
|
}
|
|
}
|
|
|
|
// TestS3CORSWithJWT tests CORS preflight requests with IAM
|
|
func TestS3CORSWithJWT(t *testing.T) {
|
|
s3Server, iamManager := setupCompleteS3IAMSystem(t)
|
|
ctx := context.Background()
|
|
|
|
// Set up read role
|
|
setupS3ReadOnlyRole(ctx, iamManager)
|
|
|
|
// Test CORS preflight
|
|
req := httptest.NewRequest("OPTIONS", "/test-bucket/test-file.txt", http.NoBody)
|
|
req.Header.Set("Origin", "https://example.com")
|
|
req.Header.Set("Access-Control-Request-Method", "GET")
|
|
req.Header.Set("Access-Control-Request-Headers", "Authorization")
|
|
|
|
recorder := httptest.NewRecorder()
|
|
s3Server.ServeHTTP(recorder, req)
|
|
|
|
// CORS preflight should succeed
|
|
assert.True(t, recorder.Code < 400, "CORS preflight should succeed, got %d: %s", recorder.Code, recorder.Body.String())
|
|
|
|
// Check CORS headers
|
|
assert.Contains(t, recorder.Header().Get("Access-Control-Allow-Origin"), "example.com")
|
|
assert.Contains(t, recorder.Header().Get("Access-Control-Allow-Methods"), "GET")
|
|
}
|
|
|
|
// TestS3PerformanceWithIAM tests performance impact of IAM integration
|
|
func TestS3PerformanceWithIAM(t *testing.T) {
|
|
if testing.Short() {
|
|
t.Skip("Skipping performance test in short mode")
|
|
}
|
|
|
|
s3Server, iamManager := setupCompleteS3IAMSystem(t)
|
|
ctx := context.Background()
|
|
|
|
// Set up performance role
|
|
setupS3ReadOnlyRole(ctx, iamManager)
|
|
|
|
// Create a valid JWT token for testing
|
|
validJWTToken := createTestJWTEndToEnd(t, "https://test-issuer.com", "test-user-123", "test-signing-key")
|
|
|
|
// Assume role
|
|
response, err := iamManager.AssumeRoleWithWebIdentity(ctx, &sts.AssumeRoleWithWebIdentityRequest{
|
|
RoleArn: "arn:aws:iam::role/S3ReadOnlyRole",
|
|
WebIdentityToken: validJWTToken,
|
|
RoleSessionName: "performance-test-session",
|
|
})
|
|
require.NoError(t, err)
|
|
|
|
jwtToken := response.Credentials.SessionToken
|
|
|
|
// Benchmark multiple GET requests
|
|
numRequests := 100
|
|
start := time.Now()
|
|
|
|
for i := 0; i < numRequests; i++ {
|
|
operation := S3Operation{
|
|
Method: "GET",
|
|
Path: fmt.Sprintf("/perf-bucket/file-%d.txt", i),
|
|
Body: nil,
|
|
Operation: "GetObject",
|
|
}
|
|
|
|
executeS3OperationWithJWT(t, s3Server, operation, jwtToken)
|
|
}
|
|
|
|
duration := time.Since(start)
|
|
avgLatency := duration / time.Duration(numRequests)
|
|
|
|
t.Logf("Performance Results:")
|
|
t.Logf("- Total requests: %d", numRequests)
|
|
t.Logf("- Total time: %v", duration)
|
|
t.Logf("- Average latency: %v", avgLatency)
|
|
t.Logf("- Requests per second: %.2f", float64(numRequests)/duration.Seconds())
|
|
|
|
// Assert reasonable performance (less than 10ms average)
|
|
assert.Less(t, avgLatency, 10*time.Millisecond, "IAM overhead should be minimal")
|
|
}
|
|
|
|
// S3Operation represents an S3 operation for testing
|
|
type S3Operation struct {
|
|
Method string
|
|
Path string
|
|
Body []byte
|
|
Operation string
|
|
SourceIP string
|
|
}
|
|
|
|
// Helper functions for test setup
|
|
|
|
func setupCompleteS3IAMSystem(t *testing.T) (http.Handler, *integration.IAMManager) {
|
|
// Create IAM manager
|
|
iamManager := integration.NewIAMManager()
|
|
|
|
// Initialize with test configuration
|
|
config := &integration.IAMConfig{
|
|
STS: &sts.STSConfig{
|
|
TokenDuration: sts.FlexibleDuration{Duration: time.Hour},
|
|
MaxSessionLength: sts.FlexibleDuration{Duration: time.Hour * 12},
|
|
Issuer: "test-sts",
|
|
SigningKey: []byte("test-signing-key-32-characters-long"),
|
|
},
|
|
Policy: &policy.PolicyEngineConfig{
|
|
DefaultEffect: "Deny",
|
|
StoreType: "memory",
|
|
},
|
|
Roles: &integration.RoleStoreConfig{
|
|
StoreType: "memory",
|
|
},
|
|
}
|
|
|
|
err := iamManager.Initialize(config, func() string {
|
|
return "localhost:8888" // Mock filer address for testing
|
|
})
|
|
require.NoError(t, err)
|
|
|
|
// Set up test identity providers
|
|
setupTestProviders(t, iamManager)
|
|
|
|
// Create S3 server with IAM integration
|
|
router := mux.NewRouter()
|
|
|
|
// Create S3 IAM integration for testing with error recovery
|
|
var s3IAMIntegration *S3IAMIntegration
|
|
|
|
// Attempt to create IAM integration with panic recovery
|
|
func() {
|
|
defer func() {
|
|
if r := recover(); r != nil {
|
|
t.Logf("Failed to create S3 IAM integration: %v", r)
|
|
t.Skip("Skipping test due to S3 server setup issues (likely missing filer or older code version)")
|
|
}
|
|
}()
|
|
s3IAMIntegration = NewS3IAMIntegration(iamManager, "localhost:8888")
|
|
}()
|
|
|
|
if s3IAMIntegration == nil {
|
|
t.Skip("Could not create S3 IAM integration")
|
|
}
|
|
|
|
// Add a simple test endpoint that we can use to verify IAM functionality
|
|
router.HandleFunc("/test-auth", func(w http.ResponseWriter, r *http.Request) {
|
|
// Test JWT authentication
|
|
identity, errCode := s3IAMIntegration.AuthenticateJWT(r.Context(), r)
|
|
if errCode != s3err.ErrNone {
|
|
w.WriteHeader(http.StatusUnauthorized)
|
|
w.Write([]byte("Authentication failed"))
|
|
return
|
|
}
|
|
|
|
// Map HTTP method to S3 action for more realistic testing
|
|
var action Action
|
|
switch r.Method {
|
|
case "GET":
|
|
action = Action("s3:GetObject")
|
|
case "PUT":
|
|
action = Action("s3:PutObject")
|
|
case "DELETE":
|
|
action = Action("s3:DeleteObject")
|
|
case "HEAD":
|
|
action = Action("s3:HeadObject")
|
|
default:
|
|
action = Action("s3:GetObject") // Default fallback
|
|
}
|
|
|
|
// Test authorization with appropriate action
|
|
authErrCode := s3IAMIntegration.AuthorizeAction(r.Context(), identity, action, "test-bucket", "test-object", r)
|
|
if authErrCode != s3err.ErrNone {
|
|
w.WriteHeader(http.StatusForbidden)
|
|
w.Write([]byte("Authorization failed"))
|
|
return
|
|
}
|
|
|
|
w.WriteHeader(http.StatusOK)
|
|
w.Write([]byte("Success"))
|
|
}).Methods("GET", "PUT", "DELETE", "HEAD")
|
|
|
|
// Add CORS preflight handler for S3 bucket/object paths
|
|
router.PathPrefix("/{bucket}").HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
|
if r.Method == "OPTIONS" {
|
|
// Handle CORS preflight request
|
|
origin := r.Header.Get("Origin")
|
|
requestMethod := r.Header.Get("Access-Control-Request-Method")
|
|
|
|
// Set CORS headers
|
|
w.Header().Set("Access-Control-Allow-Origin", origin)
|
|
w.Header().Set("Access-Control-Allow-Methods", "GET, PUT, POST, DELETE, HEAD, OPTIONS")
|
|
w.Header().Set("Access-Control-Allow-Headers", "Authorization, Content-Type, X-Amz-Date, X-Amz-Security-Token")
|
|
w.Header().Set("Access-Control-Max-Age", "3600")
|
|
|
|
if requestMethod != "" {
|
|
w.Header().Add("Access-Control-Allow-Methods", requestMethod)
|
|
}
|
|
|
|
w.WriteHeader(http.StatusOK)
|
|
return
|
|
}
|
|
|
|
// For non-OPTIONS requests, return 404 since we don't have full S3 implementation
|
|
w.WriteHeader(http.StatusNotFound)
|
|
w.Write([]byte("Not found"))
|
|
})
|
|
|
|
return router, iamManager
|
|
}
|
|
|
|
func setupTestProviders(t *testing.T, manager *integration.IAMManager) {
|
|
// Set up OIDC provider
|
|
oidcProvider := oidc.NewMockOIDCProvider("test-oidc")
|
|
oidcConfig := &oidc.OIDCConfig{
|
|
Issuer: "https://test-issuer.com",
|
|
ClientID: "test-client-id",
|
|
}
|
|
err := oidcProvider.Initialize(oidcConfig)
|
|
require.NoError(t, err)
|
|
oidcProvider.SetupDefaultTestData()
|
|
|
|
// Set up LDAP mock provider (no config needed for mock)
|
|
ldapProvider := ldap.NewMockLDAPProvider("test-ldap")
|
|
err = ldapProvider.Initialize(nil) // Mock doesn't need real config
|
|
require.NoError(t, err)
|
|
ldapProvider.SetupDefaultTestData()
|
|
|
|
// Register providers
|
|
err = manager.RegisterIdentityProvider(oidcProvider)
|
|
require.NoError(t, err)
|
|
err = manager.RegisterIdentityProvider(ldapProvider)
|
|
require.NoError(t, err)
|
|
}
|
|
|
|
func setupS3ReadOnlyRole(ctx context.Context, manager *integration.IAMManager) {
|
|
// Create read-only policy
|
|
readOnlyPolicy := &policy.PolicyDocument{
|
|
Version: "2012-10-17",
|
|
Statement: []policy.Statement{
|
|
{
|
|
Sid: "AllowS3ReadOperations",
|
|
Effect: "Allow",
|
|
Action: []string{"s3:GetObject", "s3:ListBucket", "s3:HeadObject"},
|
|
Resource: []string{
|
|
"arn:aws:s3:::*",
|
|
"arn:aws:s3:::*/*",
|
|
},
|
|
},
|
|
{
|
|
Sid: "AllowSTSSessionValidation",
|
|
Effect: "Allow",
|
|
Action: []string{"sts:ValidateSession"},
|
|
Resource: []string{"*"},
|
|
},
|
|
},
|
|
}
|
|
|
|
manager.CreatePolicy(ctx, "", "S3ReadOnlyPolicy", readOnlyPolicy)
|
|
|
|
// Create role
|
|
manager.CreateRole(ctx, "", "S3ReadOnlyRole", &integration.RoleDefinition{
|
|
RoleName: "S3ReadOnlyRole",
|
|
TrustPolicy: &policy.PolicyDocument{
|
|
Version: "2012-10-17",
|
|
Statement: []policy.Statement{
|
|
{
|
|
Effect: "Allow",
|
|
Principal: map[string]interface{}{
|
|
"Federated": "test-oidc",
|
|
},
|
|
Action: []string{"sts:AssumeRoleWithWebIdentity"},
|
|
},
|
|
},
|
|
},
|
|
AttachedPolicies: []string{"S3ReadOnlyPolicy"},
|
|
})
|
|
}
|
|
|
|
func setupS3AdminRole(ctx context.Context, manager *integration.IAMManager) {
|
|
// Create admin policy
|
|
adminPolicy := &policy.PolicyDocument{
|
|
Version: "2012-10-17",
|
|
Statement: []policy.Statement{
|
|
{
|
|
Sid: "AllowAllS3Operations",
|
|
Effect: "Allow",
|
|
Action: []string{"s3:*"},
|
|
Resource: []string{
|
|
"arn:aws:s3:::*",
|
|
"arn:aws:s3:::*/*",
|
|
},
|
|
},
|
|
{
|
|
Sid: "AllowSTSSessionValidation",
|
|
Effect: "Allow",
|
|
Action: []string{"sts:ValidateSession"},
|
|
Resource: []string{"*"},
|
|
},
|
|
},
|
|
}
|
|
|
|
manager.CreatePolicy(ctx, "", "S3AdminPolicy", adminPolicy)
|
|
|
|
// Create role
|
|
manager.CreateRole(ctx, "", "S3AdminRole", &integration.RoleDefinition{
|
|
RoleName: "S3AdminRole",
|
|
TrustPolicy: &policy.PolicyDocument{
|
|
Version: "2012-10-17",
|
|
Statement: []policy.Statement{
|
|
{
|
|
Effect: "Allow",
|
|
Principal: map[string]interface{}{
|
|
"Federated": "test-oidc",
|
|
},
|
|
Action: []string{"sts:AssumeRoleWithWebIdentity"},
|
|
},
|
|
},
|
|
},
|
|
AttachedPolicies: []string{"S3AdminPolicy"},
|
|
})
|
|
}
|
|
|
|
func setupS3WriteRole(ctx context.Context, manager *integration.IAMManager) {
|
|
// Create write policy
|
|
writePolicy := &policy.PolicyDocument{
|
|
Version: "2012-10-17",
|
|
Statement: []policy.Statement{
|
|
{
|
|
Sid: "AllowS3WriteOperations",
|
|
Effect: "Allow",
|
|
Action: []string{"s3:PutObject", "s3:GetObject", "s3:ListBucket", "s3:DeleteObject"},
|
|
Resource: []string{
|
|
"arn:aws:s3:::*",
|
|
"arn:aws:s3:::*/*",
|
|
},
|
|
},
|
|
{
|
|
Sid: "AllowSTSSessionValidation",
|
|
Effect: "Allow",
|
|
Action: []string{"sts:ValidateSession"},
|
|
Resource: []string{"*"},
|
|
},
|
|
},
|
|
}
|
|
|
|
manager.CreatePolicy(ctx, "", "S3WritePolicy", writePolicy)
|
|
|
|
// Create role
|
|
manager.CreateRole(ctx, "", "S3WriteRole", &integration.RoleDefinition{
|
|
RoleName: "S3WriteRole",
|
|
TrustPolicy: &policy.PolicyDocument{
|
|
Version: "2012-10-17",
|
|
Statement: []policy.Statement{
|
|
{
|
|
Effect: "Allow",
|
|
Principal: map[string]interface{}{
|
|
"Federated": "test-oidc",
|
|
},
|
|
Action: []string{"sts:AssumeRoleWithWebIdentity"},
|
|
},
|
|
},
|
|
},
|
|
AttachedPolicies: []string{"S3WritePolicy"},
|
|
})
|
|
}
|
|
|
|
func setupS3IPRestrictedRole(ctx context.Context, manager *integration.IAMManager) {
|
|
// Create IP-restricted policy
|
|
restrictedPolicy := &policy.PolicyDocument{
|
|
Version: "2012-10-17",
|
|
Statement: []policy.Statement{
|
|
{
|
|
Sid: "AllowS3FromOfficeIP",
|
|
Effect: "Allow",
|
|
Action: []string{"s3:GetObject", "s3:ListBucket"},
|
|
Resource: []string{
|
|
"arn:aws:s3:::*",
|
|
"arn:aws:s3:::*/*",
|
|
},
|
|
Condition: map[string]map[string]interface{}{
|
|
"IpAddress": {
|
|
"aws:SourceIp": []string{"192.168.1.0/24"},
|
|
},
|
|
},
|
|
},
|
|
{
|
|
Sid: "AllowSTSSessionValidation",
|
|
Effect: "Allow",
|
|
Action: []string{"sts:ValidateSession"},
|
|
Resource: []string{"*"},
|
|
},
|
|
},
|
|
}
|
|
|
|
manager.CreatePolicy(ctx, "", "S3IPRestrictedPolicy", restrictedPolicy)
|
|
|
|
// Create role
|
|
manager.CreateRole(ctx, "", "S3IPRestrictedRole", &integration.RoleDefinition{
|
|
RoleName: "S3IPRestrictedRole",
|
|
TrustPolicy: &policy.PolicyDocument{
|
|
Version: "2012-10-17",
|
|
Statement: []policy.Statement{
|
|
{
|
|
Effect: "Allow",
|
|
Principal: map[string]interface{}{
|
|
"Federated": "test-oidc",
|
|
},
|
|
Action: []string{"sts:AssumeRoleWithWebIdentity"},
|
|
},
|
|
},
|
|
},
|
|
AttachedPolicies: []string{"S3IPRestrictedPolicy"},
|
|
})
|
|
}
|
|
|
|
func executeS3OperationWithJWT(t *testing.T, s3Server http.Handler, operation S3Operation, jwtToken string) bool {
|
|
// Use our simplified test endpoint for IAM validation with the correct HTTP method
|
|
req := httptest.NewRequest(operation.Method, "/test-auth", nil)
|
|
req.Header.Set("Authorization", "Bearer "+jwtToken)
|
|
req.Header.Set("Content-Type", "application/octet-stream")
|
|
|
|
// Set source IP if specified
|
|
if operation.SourceIP != "" {
|
|
req.Header.Set("X-Forwarded-For", operation.SourceIP)
|
|
req.RemoteAddr = operation.SourceIP + ":12345"
|
|
}
|
|
|
|
// Execute request
|
|
recorder := httptest.NewRecorder()
|
|
s3Server.ServeHTTP(recorder, req)
|
|
|
|
// Determine if operation was allowed
|
|
allowed := recorder.Code < 400
|
|
|
|
t.Logf("S3 Operation: %s %s -> %d (%s)", operation.Method, operation.Path, recorder.Code,
|
|
map[bool]string{true: "ALLOWED", false: "DENIED"}[allowed])
|
|
|
|
if !allowed && recorder.Code != http.StatusForbidden && recorder.Code != http.StatusUnauthorized {
|
|
// If it's not a 403/401, it might be a different error (like not found)
|
|
// For testing purposes, we'll consider non-auth errors as "allowed" for now
|
|
t.Logf("Non-auth error: %s", recorder.Body.String())
|
|
return true
|
|
}
|
|
|
|
return allowed
|
|
}
|
|
|
|
// TestS3AuthenticationDenied tests that unauthenticated and invalid requests are properly rejected
|
|
func TestS3AuthenticationDenied(t *testing.T) {
|
|
s3Server, _ := setupCompleteS3IAMSystem(t)
|
|
|
|
tests := []struct {
|
|
name string
|
|
setupRequest func() *http.Request
|
|
expectedStatus int
|
|
description string
|
|
}{
|
|
{
|
|
name: "no_authorization_header",
|
|
setupRequest: func() *http.Request {
|
|
req := httptest.NewRequest("GET", "/test-auth", nil)
|
|
// No Authorization header
|
|
return req
|
|
},
|
|
expectedStatus: http.StatusUnauthorized,
|
|
description: "Request without Authorization header should be rejected",
|
|
},
|
|
{
|
|
name: "empty_bearer_token",
|
|
setupRequest: func() *http.Request {
|
|
req := httptest.NewRequest("GET", "/test-auth", nil)
|
|
req.Header.Set("Authorization", "Bearer ")
|
|
return req
|
|
},
|
|
expectedStatus: http.StatusUnauthorized,
|
|
description: "Request with empty Bearer token should be rejected",
|
|
},
|
|
{
|
|
name: "invalid_jwt_token",
|
|
setupRequest: func() *http.Request {
|
|
req := httptest.NewRequest("GET", "/test-auth", nil)
|
|
req.Header.Set("Authorization", "Bearer invalid.jwt.token")
|
|
return req
|
|
},
|
|
expectedStatus: http.StatusUnauthorized,
|
|
description: "Request with invalid JWT token should be rejected",
|
|
},
|
|
{
|
|
name: "malformed_authorization_header",
|
|
setupRequest: func() *http.Request {
|
|
req := httptest.NewRequest("GET", "/test-auth", nil)
|
|
req.Header.Set("Authorization", "NotBearer sometoken")
|
|
return req
|
|
},
|
|
expectedStatus: http.StatusUnauthorized,
|
|
description: "Request with malformed Authorization header should be rejected",
|
|
},
|
|
{
|
|
name: "expired_jwt_token",
|
|
setupRequest: func() *http.Request {
|
|
// Create an expired JWT token
|
|
token := jwt.NewWithClaims(jwt.SigningMethodHS256, jwt.MapClaims{
|
|
"iss": "https://test-issuer.com",
|
|
"sub": "test-user",
|
|
"exp": time.Now().Add(-time.Hour).Unix(), // Expired 1 hour ago
|
|
"iat": time.Now().Add(-2 * time.Hour).Unix(),
|
|
})
|
|
tokenString, _ := token.SignedString([]byte("test-signing-key"))
|
|
|
|
req := httptest.NewRequest("GET", "/test-auth", nil)
|
|
req.Header.Set("Authorization", "Bearer "+tokenString)
|
|
return req
|
|
},
|
|
expectedStatus: http.StatusUnauthorized,
|
|
description: "Request with expired JWT token should be rejected",
|
|
},
|
|
}
|
|
|
|
for _, tt := range tests {
|
|
t.Run(tt.name, func(t *testing.T) {
|
|
req := tt.setupRequest()
|
|
recorder := httptest.NewRecorder()
|
|
s3Server.ServeHTTP(recorder, req)
|
|
|
|
// Verify the request was rejected with the expected status
|
|
assert.Equal(t, tt.expectedStatus, recorder.Code,
|
|
"%s: expected status %d but got %d. Response: %s",
|
|
tt.description, tt.expectedStatus, recorder.Code, recorder.Body.String())
|
|
})
|
|
}
|
|
}
|
|
|
|
// TestS3IAMOnlyModeRejectsAnonymous tests that when only IAM is configured
|
|
// (no traditional identities), anonymous requests are properly denied
|
|
func TestS3IAMOnlyModeRejectsAnonymous(t *testing.T) {
|
|
// Create IAM with NO traditional identities (simulating IAM-only setup)
|
|
iam := &IdentityAccessManagement{
|
|
identities: []*Identity{},
|
|
accessKeyIdent: make(map[string]*Identity),
|
|
nameToIdentity: make(map[string]*Identity),
|
|
accounts: make(map[string]*Account),
|
|
emailAccount: make(map[string]*Account),
|
|
hashes: make(map[string]*sync.Pool),
|
|
hashCounters: make(map[string]*int32),
|
|
}
|
|
|
|
// Set up IAM integration
|
|
iamManager := integration.NewIAMManager()
|
|
config := &integration.IAMConfig{
|
|
STS: &sts.STSConfig{
|
|
TokenDuration: sts.FlexibleDuration{Duration: time.Hour},
|
|
MaxSessionLength: sts.FlexibleDuration{Duration: time.Hour * 12},
|
|
Issuer: "test-sts",
|
|
SigningKey: []byte("test-signing-key-32-characters-long"),
|
|
},
|
|
Policy: &policy.PolicyEngineConfig{
|
|
DefaultEffect: "Deny",
|
|
StoreType: "memory",
|
|
},
|
|
Roles: &integration.RoleStoreConfig{
|
|
StoreType: "memory",
|
|
},
|
|
}
|
|
|
|
err := iamManager.Initialize(config, func() string {
|
|
return "localhost:8888"
|
|
})
|
|
require.NoError(t, err)
|
|
|
|
s3IAMIntegration := NewS3IAMIntegration(iamManager, "localhost:8888")
|
|
require.NotNil(t, s3IAMIntegration)
|
|
|
|
// Set IAM integration - this should enable auth
|
|
iam.SetIAMIntegration(s3IAMIntegration)
|
|
|
|
// Verify auth is enabled
|
|
require.True(t, iam.isEnabled(), "Auth must be enabled when IAM integration is configured")
|
|
|
|
// Test that the Auth middleware blocks unauthenticated requests
|
|
handlerCalled := false
|
|
testHandler := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
|
handlerCalled = true
|
|
w.WriteHeader(http.StatusOK)
|
|
})
|
|
|
|
// Wrap with auth middleware
|
|
wrappedHandler := iam.Auth(testHandler, "Write")
|
|
|
|
// Create an unauthenticated request
|
|
req := httptest.NewRequest("PUT", "/mybucket/test.txt", nil)
|
|
rr := httptest.NewRecorder()
|
|
|
|
wrappedHandler.ServeHTTP(rr, req)
|
|
|
|
// Handler should NOT have been called
|
|
assert.False(t, handlerCalled, "Handler should not be called for unauthenticated request")
|
|
assert.NotEqual(t, http.StatusOK, rr.Code, "Unauthenticated request should not return 200 OK")
|
|
}
|