* feat: Add AWS IAM Policy Variables support to S3 API
Implements policy variables for dynamic access control in bucket policies.
Supported variables:
- aws:username - Extracted from principal ARN
- aws:userid - User identifier (same as username in SeaweedFS)
- aws:principaltype - IAMUser, IAMRole, or AssumedRole
- jwt:* - Any JWT claim (e.g., jwt:preferred_username, jwt:sub)
Key changes:
- Added PolicyVariableRegex to detect ${...} patterns
- Extended CompiledStatement with DynamicResourcePatterns, DynamicPrincipalPatterns, DynamicActionPatterns
- Added Claims field to PolicyEvaluationArgs for JWT claim access
- Implemented SubstituteVariables() for variable replacement from context and JWT claims
- Implemented extractPrincipalVariables() for ARN parsing
- Updated EvaluateConditions() to support variable substitution
- Comprehensive unit and integration tests
Resolves #8037
* feat: Add LDAP and PrincipalAccount variable support
Completes future enhancements for policy variables:
- Added ldap:* variable support for LDAP claims
- ldap:username - LDAP username from claims
- ldap:dn - LDAP distinguished name from claims
- ldap:* - Any LDAP claim
- Added aws:PrincipalAccount extraction from ARN
- Extracts account ID from principal ARN
- Available as ${aws:PrincipalAccount} in policies
Updated SubstituteVariables() to check LDAP claims
Updated extractPrincipalVariables() to extract account ID
Added comprehensive tests for new variables
* feat(s3api): implement IAM policy variables core logic and optimization
* feat(s3api): integrate policy variables with S3 authentication and handlers
* test(s3api): add integration tests for policy variables
* cleanup: remove unused policy conversion files
* Add S3 policy variables integration tests and path support
- Add comprehensive integration tests for policy variables
- Test username isolation, JWT claims, LDAP claims
- Add support for IAM paths in principal ARN parsing
- Add tests for principals with paths
* Fix IAM Role principal variable extraction
IAM Roles should not have aws:userid or aws:PrincipalAccount
according to AWS behavior. Only IAM Users and Assumed Roles
should have these variables.
Fixes TestExtractPrincipalVariables test failures.
* Security fixes and bug fixes for S3 policy variables
SECURITY FIXES:
- Prevent X-SeaweedFS-Principal header spoofing by clearing internal
headers at start of authentication (auth_credentials.go)
- Restrict policy variable substitution to safe allowlist to prevent
client header injection (iam/policy/policy_engine.go)
- Add core policy validation before storing bucket policies
BUG FIXES:
- Remove unused sid variable in evaluateStatement
- Fix LDAP claim lookup to check both prefixed and unprefixed keys
- Add ValidatePolicy call in PutBucketPolicyHandler
These fixes prevent privilege escalation via header injection and
ensure only validated identity claims are used in policy evaluation.
* Additional security fixes and code cleanup
SECURITY FIXES:
- Fixed X-Forwarded-For spoofing by only trusting proxy headers from
private/localhost IPs (s3_iam_middleware.go)
- Changed context key from "sourceIP" to "aws:SourceIp" for proper
policy variable substitution
CODE IMPROVEMENTS:
- Kept aws:PrincipalAccount for IAM Roles to support condition evaluations
- Removed redundant STS principaltype override
- Removed unused service variable
- Cleaned up commented-out debug logging statements
- Updated tests to reflect new IAM Role behavior
These changes prevent IP spoofing attacks and ensure policy variables
work correctly with the safe allowlist.
* Add security documentation for ParseJWTToken
Added comprehensive security comments explaining that ParseJWTToken
is safe despite parsing without verification because:
- It's only used for routing to the correct verification method
- All code paths perform cryptographic verification before trusting claims
- OIDC tokens: validated via validateExternalOIDCToken
- STS tokens: validated via ValidateSessionToken
Enhanced function documentation with clear security warnings about
proper usage to prevent future misuse.
* Fix IP condition evaluation to use aws:SourceIp key
Fixed evaluateIPCondition in IAM policy engine to use "aws:SourceIp"
instead of "sourceIP" to match the updated extractRequestContext.
This fixes the failing IP-restricted role test where IP-based policy
conditions were not being evaluated correctly.
Updated all test cases to use the correct "aws:SourceIp" key.
* Address code review feedback: optimize and clarify
PERFORMANCE IMPROVEMENT:
- Optimized expandPolicyVariables to use regexp.ReplaceAllStringFunc
for single-pass variable substitution instead of iterating through
all safe variables. This improves performance from O(n*m) to O(m)
where n is the number of safe variables and m is the pattern length.
CODE CLARITY:
- Added detailed comment explaining LDAP claim fallback mechanism
(checks both prefixed and unprefixed keys for compatibility)
- Enhanced TODO comment for trusted proxy configuration with rationale
and recommendations for supporting cloud load balancers, CDNs, and
complex network topologies
All tests passing.
* Address Copilot code review feedback
BUG FIXES:
- Fixed type switch for int/int32/int64 - separated into individual cases
since interface type switches only match the first type in multi-type cases
- Fixed grammatically incorrect error message in types.go
CODE QUALITY:
- Removed duplicate Resource/NotResource validation (already in ValidateStatement)
- Added comprehensive comment explaining isEnabled() logic and security implications
- Improved trusted proxy NOTE comment to be more concise while noting limitations
All tests passing.
* Fix test failures after extractSourceIP security changes
Updated tests to work with the security fix that only trusts
X-Forwarded-For/X-Real-IP headers from private IP addresses:
- Set RemoteAddr to 127.0.0.1 in tests to simulate trusted proxy
- Changed context key from "sourceIP" to "aws:SourceIp"
- Added test case for untrusted proxy (public RemoteAddr)
- Removed invalid ValidateStatement call (validation happens in ValidatePolicy)
All tests now passing.
* Address remaining Gemini code review feedback
CODE SAFETY:
- Deep clone Action field in CompileStatement to prevent potential data races
if the original policy document is modified after compilation
TEST CLEANUP:
- Remove debug logging (fmt.Fprintf) from engine_notresource_test.go
- Remove unused imports in engine_notresource_test.go
All tests passing.
* Fix insecure JWT parsing in IAM auth flow
SECURITY FIX:
- Renamed ParseJWTToken to ParseUnverifiedJWTToken with explicit security warnings.
- Refactored AuthenticateJWT to use the trusted SessionInfo returned by ValidateSessionToken
instead of relying on unverified claims from the initial parse.
- Refactored ValidatePresignedURLWithIAM to reuse the robust AuthenticateJWT logic, removing
duplicated and insecure manual token parsing.
This ensures all identity information (Role, Principal, Subject) used for authorization
decisions is derived solely from cryptographically verified tokens.
* Security: Fix insecure JWT claim extraction in policy engine
- Refactored EvaluatePolicy to accept trusted claims from verified Identity instead of parsing unverified tokens
- Updated AuthenticateJWT to populate Claims in IAMIdentity from verified sources (SessionInfo/ExternalIdentity)
- Updated s3api_server and handlers to pass claims correctly
- Improved isPrivateIP to support IPv6 loopback, link-local, and ULA
- Fixed flaky distributed_session_consistency test with retry logic
* fix(iam): populate Subject in STSSessionInfo to ensure correct identity propagation
This fixes the TestS3IAMAuthentication/valid_jwt_token_authentication failure by ensuring the session subject (sub) is correctly mapped to the internal SessionInfo struct, allowing bucket ownership validation to succeed.
* Optimized isPrivateIP
* Create s3-policy-tests.yml
* fix tests
* fix tests
* tests(s3/iam): simplify policy to resource-based \ (step 1)
* tests(s3/iam): add explicit Deny NotResource for isolation (step 2)
* fixes
* policy: skip resource matching for STS trust policies to allow AssumeRole evaluation
* refactor: remove debug logging and hoist policy variables for performance
* test: fix TestS3IAMBucketPolicyIntegration cleanup to handle per-subtest object lifecycle
* test: fix bucket name generation to comply with S3 63-char limit
* test: skip TestS3IAMPolicyEnforcement until role setup is implemented
* test: use weed mini for simpler test server deployment
Replace 'weed server' with 'weed mini' for IAM tests to avoid port binding issues
and simplify the all-in-one server deployment. This improves test reliability
and execution time.
* security: prevent allocation overflow in policy evaluation
Add maxPoliciesForEvaluation constant to cap the number of policies evaluated
in a single request. This prevents potential integer overflow when allocating
slices for policy lists that may be influenced by untrusted input.
Changes:
- Add const maxPoliciesForEvaluation = 1024 to set an upper bound
- Validate len(policies) < maxPoliciesForEvaluation before appending bucket policy
- Use append() instead of make([]string, len+1) to avoid arithmetic overflow
- Apply fix to both IsActionAllowed policy evaluation paths
176 lines
5.6 KiB
Go
176 lines
5.6 KiB
Go
package sts
|
|
|
|
import (
|
|
"fmt"
|
|
"time"
|
|
|
|
"github.com/golang-jwt/jwt/v5"
|
|
"github.com/seaweedfs/seaweedfs/weed/glog"
|
|
)
|
|
|
|
// defaultCredentialGenerator is a reusable instance for generating temporary credentials
|
|
// Reusing a single instance across all calls to ToSessionInfo() reduces allocation overhead
|
|
// since this method may be called frequently during signature verification
|
|
var defaultCredentialGenerator = NewCredentialGenerator()
|
|
|
|
// STSSessionClaims represents comprehensive session information embedded in JWT tokens
|
|
// This eliminates the need for separate session storage by embedding all session
|
|
// metadata directly in the token itself - enabling true stateless operation
|
|
type STSSessionClaims struct {
|
|
jwt.RegisteredClaims
|
|
|
|
// Session identification
|
|
SessionId string `json:"sid"` // session_id (abbreviated for smaller tokens)
|
|
SessionName string `json:"snam"` // session_name (abbreviated for smaller tokens)
|
|
TokenType string `json:"typ"` // token_type
|
|
|
|
// Role information
|
|
RoleArn string `json:"role"` // role_arn
|
|
AssumedRole string `json:"assumed"` // assumed_role_user
|
|
Principal string `json:"principal"` // principal_arn
|
|
|
|
// Authorization data
|
|
Policies []string `json:"pol,omitempty"` // policies (abbreviated)
|
|
|
|
// Identity provider information
|
|
IdentityProvider string `json:"idp"` // identity_provider
|
|
ExternalUserId string `json:"ext_uid"` // external_user_id
|
|
ProviderIssuer string `json:"prov_iss"` // provider_issuer
|
|
|
|
// Request context (optional, for policy evaluation)
|
|
RequestContext map[string]interface{} `json:"req_ctx,omitempty"`
|
|
|
|
// Session metadata
|
|
AssumedAt time.Time `json:"assumed_at"` // when role was assumed
|
|
MaxDuration int64 `json:"max_dur,omitempty"` // maximum session duration in seconds
|
|
}
|
|
|
|
// NewSTSSessionClaims creates new STS session claims with all required information
|
|
func NewSTSSessionClaims(sessionId, issuer string, expiresAt time.Time) *STSSessionClaims {
|
|
now := time.Now()
|
|
return &STSSessionClaims{
|
|
RegisteredClaims: jwt.RegisteredClaims{
|
|
Issuer: issuer,
|
|
Subject: sessionId,
|
|
IssuedAt: jwt.NewNumericDate(now),
|
|
ExpiresAt: jwt.NewNumericDate(expiresAt),
|
|
NotBefore: jwt.NewNumericDate(now),
|
|
},
|
|
SessionId: sessionId,
|
|
TokenType: TokenTypeSession,
|
|
AssumedAt: now,
|
|
}
|
|
}
|
|
|
|
// ToSessionInfo converts JWT claims back to SessionInfo structure
|
|
// This enables seamless integration with existing code expecting SessionInfo
|
|
func (c *STSSessionClaims) ToSessionInfo() *SessionInfo {
|
|
var expiresAt time.Time
|
|
if c.ExpiresAt != nil {
|
|
expiresAt = c.ExpiresAt.Time
|
|
}
|
|
|
|
// Generate temporary credentials from the session ID
|
|
// This is deterministic based on the session ID, so the same credentials are regenerated
|
|
credentials, err := defaultCredentialGenerator.GenerateTemporaryCredentials(c.SessionId, expiresAt)
|
|
if err != nil {
|
|
// Log the error with context - credential generation failure is important for debugging
|
|
errMsg := fmt.Errorf("generate temporary credentials for session %s: %w", c.SessionId, err)
|
|
glog.Warningf("Failed to generate credentials for STS session: %v", errMsg)
|
|
// Return session info without credentials - validation will catch this as invalid
|
|
credentials = nil
|
|
}
|
|
|
|
return &SessionInfo{
|
|
SessionId: c.SessionId,
|
|
SessionName: c.SessionName,
|
|
RoleArn: c.RoleArn,
|
|
AssumedRoleUser: c.AssumedRole,
|
|
Principal: c.Principal,
|
|
Policies: c.Policies,
|
|
ExpiresAt: expiresAt,
|
|
IdentityProvider: c.IdentityProvider,
|
|
ExternalUserId: c.ExternalUserId,
|
|
ProviderIssuer: c.ProviderIssuer,
|
|
RequestContext: c.RequestContext,
|
|
// Provide the Subject (sub) from registered claims
|
|
Subject: c.Subject,
|
|
Credentials: credentials,
|
|
}
|
|
}
|
|
|
|
// IsValid checks if the session claims are valid (not expired, etc.)
|
|
func (c *STSSessionClaims) IsValid() bool {
|
|
now := time.Now()
|
|
|
|
// Check expiration
|
|
if c.ExpiresAt != nil && c.ExpiresAt.Before(now) {
|
|
return false
|
|
}
|
|
|
|
// Check not-before
|
|
if c.NotBefore != nil && c.NotBefore.After(now) {
|
|
return false
|
|
}
|
|
|
|
// Ensure required fields are present
|
|
if c.SessionId == "" || c.RoleArn == "" || c.Principal == "" {
|
|
return false
|
|
}
|
|
|
|
return true
|
|
}
|
|
|
|
// GetSessionId returns the session identifier
|
|
func (c *STSSessionClaims) GetSessionId() string {
|
|
return c.SessionId
|
|
}
|
|
|
|
// GetExpiresAt returns the expiration time
|
|
func (c *STSSessionClaims) GetExpiresAt() time.Time {
|
|
if c.ExpiresAt != nil {
|
|
return c.ExpiresAt.Time
|
|
}
|
|
return time.Time{}
|
|
}
|
|
|
|
// WithRoleInfo sets role-related information in the claims
|
|
func (c *STSSessionClaims) WithRoleInfo(roleArn, assumedRole, principal string) *STSSessionClaims {
|
|
c.RoleArn = roleArn
|
|
c.AssumedRole = assumedRole
|
|
c.Principal = principal
|
|
return c
|
|
}
|
|
|
|
// WithPolicies sets the policies associated with this session
|
|
func (c *STSSessionClaims) WithPolicies(policies []string) *STSSessionClaims {
|
|
c.Policies = policies
|
|
return c
|
|
}
|
|
|
|
// WithIdentityProvider sets identity provider information
|
|
func (c *STSSessionClaims) WithIdentityProvider(providerName, externalUserId, providerIssuer string) *STSSessionClaims {
|
|
c.IdentityProvider = providerName
|
|
c.ExternalUserId = externalUserId
|
|
c.ProviderIssuer = providerIssuer
|
|
return c
|
|
}
|
|
|
|
// WithRequestContext sets request context for policy evaluation
|
|
func (c *STSSessionClaims) WithRequestContext(ctx map[string]interface{}) *STSSessionClaims {
|
|
c.RequestContext = ctx
|
|
return c
|
|
}
|
|
|
|
// WithMaxDuration sets the maximum session duration
|
|
func (c *STSSessionClaims) WithMaxDuration(duration time.Duration) *STSSessionClaims {
|
|
c.MaxDuration = int64(duration.Seconds())
|
|
return c
|
|
}
|
|
|
|
// WithSessionName sets the session name
|
|
func (c *STSSessionClaims) WithSessionName(sessionName string) *STSSessionClaims {
|
|
c.SessionName = sessionName
|
|
return c
|
|
}
|