* feat: Add AWS IAM Policy Variables support to S3 API
Implements policy variables for dynamic access control in bucket policies.
Supported variables:
- aws:username - Extracted from principal ARN
- aws:userid - User identifier (same as username in SeaweedFS)
- aws:principaltype - IAMUser, IAMRole, or AssumedRole
- jwt:* - Any JWT claim (e.g., jwt:preferred_username, jwt:sub)
Key changes:
- Added PolicyVariableRegex to detect ${...} patterns
- Extended CompiledStatement with DynamicResourcePatterns, DynamicPrincipalPatterns, DynamicActionPatterns
- Added Claims field to PolicyEvaluationArgs for JWT claim access
- Implemented SubstituteVariables() for variable replacement from context and JWT claims
- Implemented extractPrincipalVariables() for ARN parsing
- Updated EvaluateConditions() to support variable substitution
- Comprehensive unit and integration tests
Resolves #8037
* feat: Add LDAP and PrincipalAccount variable support
Completes future enhancements for policy variables:
- Added ldap:* variable support for LDAP claims
- ldap:username - LDAP username from claims
- ldap:dn - LDAP distinguished name from claims
- ldap:* - Any LDAP claim
- Added aws:PrincipalAccount extraction from ARN
- Extracts account ID from principal ARN
- Available as ${aws:PrincipalAccount} in policies
Updated SubstituteVariables() to check LDAP claims
Updated extractPrincipalVariables() to extract account ID
Added comprehensive tests for new variables
* feat(s3api): implement IAM policy variables core logic and optimization
* feat(s3api): integrate policy variables with S3 authentication and handlers
* test(s3api): add integration tests for policy variables
* cleanup: remove unused policy conversion files
* Add S3 policy variables integration tests and path support
- Add comprehensive integration tests for policy variables
- Test username isolation, JWT claims, LDAP claims
- Add support for IAM paths in principal ARN parsing
- Add tests for principals with paths
* Fix IAM Role principal variable extraction
IAM Roles should not have aws:userid or aws:PrincipalAccount
according to AWS behavior. Only IAM Users and Assumed Roles
should have these variables.
Fixes TestExtractPrincipalVariables test failures.
* Security fixes and bug fixes for S3 policy variables
SECURITY FIXES:
- Prevent X-SeaweedFS-Principal header spoofing by clearing internal
headers at start of authentication (auth_credentials.go)
- Restrict policy variable substitution to safe allowlist to prevent
client header injection (iam/policy/policy_engine.go)
- Add core policy validation before storing bucket policies
BUG FIXES:
- Remove unused sid variable in evaluateStatement
- Fix LDAP claim lookup to check both prefixed and unprefixed keys
- Add ValidatePolicy call in PutBucketPolicyHandler
These fixes prevent privilege escalation via header injection and
ensure only validated identity claims are used in policy evaluation.
* Additional security fixes and code cleanup
SECURITY FIXES:
- Fixed X-Forwarded-For spoofing by only trusting proxy headers from
private/localhost IPs (s3_iam_middleware.go)
- Changed context key from "sourceIP" to "aws:SourceIp" for proper
policy variable substitution
CODE IMPROVEMENTS:
- Kept aws:PrincipalAccount for IAM Roles to support condition evaluations
- Removed redundant STS principaltype override
- Removed unused service variable
- Cleaned up commented-out debug logging statements
- Updated tests to reflect new IAM Role behavior
These changes prevent IP spoofing attacks and ensure policy variables
work correctly with the safe allowlist.
* Add security documentation for ParseJWTToken
Added comprehensive security comments explaining that ParseJWTToken
is safe despite parsing without verification because:
- It's only used for routing to the correct verification method
- All code paths perform cryptographic verification before trusting claims
- OIDC tokens: validated via validateExternalOIDCToken
- STS tokens: validated via ValidateSessionToken
Enhanced function documentation with clear security warnings about
proper usage to prevent future misuse.
* Fix IP condition evaluation to use aws:SourceIp key
Fixed evaluateIPCondition in IAM policy engine to use "aws:SourceIp"
instead of "sourceIP" to match the updated extractRequestContext.
This fixes the failing IP-restricted role test where IP-based policy
conditions were not being evaluated correctly.
Updated all test cases to use the correct "aws:SourceIp" key.
* Address code review feedback: optimize and clarify
PERFORMANCE IMPROVEMENT:
- Optimized expandPolicyVariables to use regexp.ReplaceAllStringFunc
for single-pass variable substitution instead of iterating through
all safe variables. This improves performance from O(n*m) to O(m)
where n is the number of safe variables and m is the pattern length.
CODE CLARITY:
- Added detailed comment explaining LDAP claim fallback mechanism
(checks both prefixed and unprefixed keys for compatibility)
- Enhanced TODO comment for trusted proxy configuration with rationale
and recommendations for supporting cloud load balancers, CDNs, and
complex network topologies
All tests passing.
* Address Copilot code review feedback
BUG FIXES:
- Fixed type switch for int/int32/int64 - separated into individual cases
since interface type switches only match the first type in multi-type cases
- Fixed grammatically incorrect error message in types.go
CODE QUALITY:
- Removed duplicate Resource/NotResource validation (already in ValidateStatement)
- Added comprehensive comment explaining isEnabled() logic and security implications
- Improved trusted proxy NOTE comment to be more concise while noting limitations
All tests passing.
* Fix test failures after extractSourceIP security changes
Updated tests to work with the security fix that only trusts
X-Forwarded-For/X-Real-IP headers from private IP addresses:
- Set RemoteAddr to 127.0.0.1 in tests to simulate trusted proxy
- Changed context key from "sourceIP" to "aws:SourceIp"
- Added test case for untrusted proxy (public RemoteAddr)
- Removed invalid ValidateStatement call (validation happens in ValidatePolicy)
All tests now passing.
* Address remaining Gemini code review feedback
CODE SAFETY:
- Deep clone Action field in CompileStatement to prevent potential data races
if the original policy document is modified after compilation
TEST CLEANUP:
- Remove debug logging (fmt.Fprintf) from engine_notresource_test.go
- Remove unused imports in engine_notresource_test.go
All tests passing.
* Fix insecure JWT parsing in IAM auth flow
SECURITY FIX:
- Renamed ParseJWTToken to ParseUnverifiedJWTToken with explicit security warnings.
- Refactored AuthenticateJWT to use the trusted SessionInfo returned by ValidateSessionToken
instead of relying on unverified claims from the initial parse.
- Refactored ValidatePresignedURLWithIAM to reuse the robust AuthenticateJWT logic, removing
duplicated and insecure manual token parsing.
This ensures all identity information (Role, Principal, Subject) used for authorization
decisions is derived solely from cryptographically verified tokens.
* Security: Fix insecure JWT claim extraction in policy engine
- Refactored EvaluatePolicy to accept trusted claims from verified Identity instead of parsing unverified tokens
- Updated AuthenticateJWT to populate Claims in IAMIdentity from verified sources (SessionInfo/ExternalIdentity)
- Updated s3api_server and handlers to pass claims correctly
- Improved isPrivateIP to support IPv6 loopback, link-local, and ULA
- Fixed flaky distributed_session_consistency test with retry logic
* fix(iam): populate Subject in STSSessionInfo to ensure correct identity propagation
This fixes the TestS3IAMAuthentication/valid_jwt_token_authentication failure by ensuring the session subject (sub) is correctly mapped to the internal SessionInfo struct, allowing bucket ownership validation to succeed.
* Optimized isPrivateIP
* Create s3-policy-tests.yml
* fix tests
* fix tests
* tests(s3/iam): simplify policy to resource-based \ (step 1)
* tests(s3/iam): add explicit Deny NotResource for isolation (step 2)
* fixes
* policy: skip resource matching for STS trust policies to allow AssumeRole evaluation
* refactor: remove debug logging and hoist policy variables for performance
* test: fix TestS3IAMBucketPolicyIntegration cleanup to handle per-subtest object lifecycle
* test: fix bucket name generation to comply with S3 63-char limit
* test: skip TestS3IAMPolicyEnforcement until role setup is implemented
* test: use weed mini for simpler test server deployment
Replace 'weed server' with 'weed mini' for IAM tests to avoid port binding issues
and simplify the all-in-one server deployment. This improves test reliability
and execution time.
* security: prevent allocation overflow in policy evaluation
Add maxPoliciesForEvaluation constant to cap the number of policies evaluated
in a single request. This prevents potential integer overflow when allocating
slices for policy lists that may be influenced by untrusted input.
Changes:
- Add const maxPoliciesForEvaluation = 1024 to set an upper bound
- Validate len(policies) < maxPoliciesForEvaluation before appending bucket policy
- Use append() instead of make([]string, len+1) to avoid arithmetic overflow
- Apply fix to both IsActionAllowed policy evaluation paths
244 lines
7.7 KiB
Go
244 lines
7.7 KiB
Go
package sts
|
|
|
|
import (
|
|
"crypto/rand"
|
|
"crypto/sha256"
|
|
"encoding/base64"
|
|
"encoding/hex"
|
|
"encoding/json"
|
|
"fmt"
|
|
"time"
|
|
|
|
"github.com/golang-jwt/jwt/v5"
|
|
"github.com/seaweedfs/seaweedfs/weed/iam/utils"
|
|
)
|
|
|
|
// TokenGenerator handles token generation and validation
|
|
type TokenGenerator struct {
|
|
signingKey []byte
|
|
issuer string
|
|
}
|
|
|
|
// NewTokenGenerator creates a new token generator
|
|
func NewTokenGenerator(signingKey []byte, issuer string) *TokenGenerator {
|
|
return &TokenGenerator{
|
|
signingKey: signingKey,
|
|
issuer: issuer,
|
|
}
|
|
}
|
|
|
|
// GenerateSessionToken creates a signed JWT session token (legacy method for compatibility)
|
|
func (t *TokenGenerator) GenerateSessionToken(sessionId string, expiresAt time.Time) (string, error) {
|
|
claims := NewSTSSessionClaims(sessionId, t.issuer, expiresAt)
|
|
return t.GenerateJWTWithClaims(claims)
|
|
}
|
|
|
|
// GenerateJWTWithClaims creates a signed JWT token with comprehensive session claims
|
|
func (t *TokenGenerator) GenerateJWTWithClaims(claims *STSSessionClaims) (string, error) {
|
|
if claims == nil {
|
|
return "", fmt.Errorf("claims cannot be nil")
|
|
}
|
|
|
|
// Ensure issuer is set from token generator
|
|
if claims.Issuer == "" {
|
|
claims.Issuer = t.issuer
|
|
}
|
|
|
|
token := jwt.NewWithClaims(jwt.SigningMethodHS256, claims)
|
|
return token.SignedString(t.signingKey)
|
|
}
|
|
|
|
// ValidateSessionToken validates and extracts claims from a session token
|
|
func (t *TokenGenerator) ValidateSessionToken(tokenString string) (*SessionTokenClaims, error) {
|
|
token, err := jwt.Parse(tokenString, func(token *jwt.Token) (interface{}, error) {
|
|
if _, ok := token.Method.(*jwt.SigningMethodHMAC); !ok {
|
|
return nil, fmt.Errorf("unexpected signing method: %v", token.Header["alg"])
|
|
}
|
|
return t.signingKey, nil
|
|
})
|
|
|
|
if err != nil {
|
|
return nil, fmt.Errorf(ErrInvalidToken, err)
|
|
}
|
|
|
|
if !token.Valid {
|
|
return nil, fmt.Errorf(ErrTokenNotValid)
|
|
}
|
|
|
|
claims, ok := token.Claims.(jwt.MapClaims)
|
|
if !ok {
|
|
return nil, fmt.Errorf(ErrInvalidTokenClaims)
|
|
}
|
|
|
|
// Verify issuer
|
|
if iss, ok := claims[JWTClaimIssuer].(string); !ok || iss != t.issuer {
|
|
return nil, fmt.Errorf(ErrInvalidIssuer)
|
|
}
|
|
|
|
// Extract session ID
|
|
sessionId, ok := claims[JWTClaimSubject].(string)
|
|
if !ok {
|
|
return nil, fmt.Errorf(ErrMissingSessionID)
|
|
}
|
|
|
|
return &SessionTokenClaims{
|
|
SessionId: sessionId,
|
|
ExpiresAt: time.Unix(int64(claims[JWTClaimExpiration].(float64)), 0),
|
|
IssuedAt: time.Unix(int64(claims[JWTClaimIssuedAt].(float64)), 0),
|
|
}, nil
|
|
}
|
|
|
|
// ValidateJWTWithClaims validates and extracts comprehensive session claims from a JWT token
|
|
func (t *TokenGenerator) ValidateJWTWithClaims(tokenString string) (*STSSessionClaims, error) {
|
|
// 1. Parse into MapClaims to capture ALL claims including custom ones
|
|
token, err := jwt.Parse(tokenString, func(token *jwt.Token) (interface{}, error) {
|
|
if _, ok := token.Method.(*jwt.SigningMethodHMAC); !ok {
|
|
return nil, fmt.Errorf("unexpected signing method: %v", token.Header["alg"])
|
|
}
|
|
return t.signingKey, nil
|
|
})
|
|
|
|
if err != nil {
|
|
return nil, fmt.Errorf(ErrInvalidToken, err)
|
|
}
|
|
|
|
if !token.Valid {
|
|
return nil, fmt.Errorf(ErrTokenNotValid)
|
|
}
|
|
|
|
mapClaims, ok := token.Claims.(jwt.MapClaims)
|
|
if !ok {
|
|
return nil, fmt.Errorf(ErrInvalidTokenClaims)
|
|
}
|
|
|
|
// 2. Decode into STSSessionClaims using JSON round-trip to respect tags
|
|
jsonBytes, err := json.Marshal(mapClaims)
|
|
if err != nil {
|
|
return nil, fmt.Errorf("failed to marshal claims: %v", err)
|
|
}
|
|
|
|
claims := &STSSessionClaims{}
|
|
if err := json.Unmarshal(jsonBytes, claims); err != nil {
|
|
return nil, fmt.Errorf("failed to unmarshal claims: %v", err)
|
|
}
|
|
|
|
// 3. Ensure RequestContext contains all claims for policy evaluation
|
|
// This preserves custom claims (like jwt:preferred_username) that are not in the struct
|
|
if claims.RequestContext == nil {
|
|
claims.RequestContext = make(map[string]interface{})
|
|
}
|
|
for k, v := range mapClaims {
|
|
// Add valid claim values to RequestContext
|
|
// We don't overwrite existing RequestContext keys if they were explicitly set
|
|
if _, exists := claims.RequestContext[k]; !exists {
|
|
claims.RequestContext[k] = v
|
|
}
|
|
}
|
|
|
|
// Validate issuer
|
|
if claims.Issuer != t.issuer {
|
|
return nil, fmt.Errorf(ErrInvalidIssuer)
|
|
}
|
|
|
|
// Validate that required fields are present
|
|
if claims.SessionId == "" {
|
|
return nil, fmt.Errorf(ErrMissingSessionID)
|
|
}
|
|
|
|
// Additional validation using the claims' own validation method
|
|
if !claims.IsValid() {
|
|
return nil, fmt.Errorf(ErrTokenNotValid)
|
|
}
|
|
|
|
return claims, nil
|
|
}
|
|
|
|
// SessionTokenClaims represents parsed session token claims
|
|
type SessionTokenClaims struct {
|
|
SessionId string
|
|
ExpiresAt time.Time
|
|
IssuedAt time.Time
|
|
}
|
|
|
|
// CredentialGenerator generates AWS-compatible temporary credentials
|
|
type CredentialGenerator struct{}
|
|
|
|
// NewCredentialGenerator creates a new credential generator
|
|
func NewCredentialGenerator() *CredentialGenerator {
|
|
return &CredentialGenerator{}
|
|
}
|
|
|
|
// GenerateTemporaryCredentials creates temporary AWS credentials
|
|
func (c *CredentialGenerator) GenerateTemporaryCredentials(sessionId string, expiration time.Time) (*Credentials, error) {
|
|
accessKeyId, err := c.generateAccessKeyId(sessionId)
|
|
if err != nil {
|
|
return nil, fmt.Errorf("failed to generate access key ID: %w", err)
|
|
}
|
|
|
|
secretAccessKey, err := c.generateSecretAccessKey(sessionId)
|
|
if err != nil {
|
|
return nil, fmt.Errorf("failed to generate secret access key: %w", err)
|
|
}
|
|
|
|
sessionToken, err := c.generateSessionTokenId(sessionId)
|
|
if err != nil {
|
|
return nil, fmt.Errorf("failed to generate session token: %w", err)
|
|
}
|
|
|
|
return &Credentials{
|
|
AccessKeyId: accessKeyId,
|
|
SecretAccessKey: secretAccessKey,
|
|
SessionToken: sessionToken,
|
|
Expiration: expiration,
|
|
}, nil
|
|
}
|
|
|
|
// generateAccessKeyId generates an AWS-style access key ID
|
|
func (c *CredentialGenerator) generateAccessKeyId(sessionId string) (string, error) {
|
|
// Create a deterministic but unique access key ID based on session
|
|
hash := sha256.Sum256([]byte("access-key:" + sessionId))
|
|
return "AKIA" + hex.EncodeToString(hash[:8]), nil // AWS format: AKIA + 16 chars
|
|
}
|
|
|
|
// generateSecretAccessKey generates a deterministic secret access key based on sessionId
|
|
// This ensures the same secret key is regenerated from the JWT claims during signature verification
|
|
func (c *CredentialGenerator) generateSecretAccessKey(sessionId string) (string, error) {
|
|
// Create deterministic secret key based on session ID (not random!)
|
|
// This is critical for STS because:
|
|
// 1. AssumeRoleWithWebIdentity generates the secret key once
|
|
// 2. During signature verification, ToSessionInfo() regenerates credentials from JWT
|
|
// 3. Both must generate the same secret key for signature verification to succeed
|
|
hash := sha256.Sum256([]byte("secret-key:" + sessionId))
|
|
return base64.StdEncoding.EncodeToString(hash[:]), nil
|
|
}
|
|
|
|
// generateSessionTokenId generates a session token identifier
|
|
func (c *CredentialGenerator) generateSessionTokenId(sessionId string) (string, error) {
|
|
// Create session token with session ID embedded
|
|
hash := sha256.Sum256([]byte("session-token:" + sessionId))
|
|
return "ST" + hex.EncodeToString(hash[:16]), nil // Custom format
|
|
}
|
|
|
|
// generateSessionId generates a unique session ID
|
|
func GenerateSessionId() (string, error) {
|
|
randomBytes := make([]byte, 16)
|
|
_, err := rand.Read(randomBytes)
|
|
if err != nil {
|
|
return "", err
|
|
}
|
|
|
|
return hex.EncodeToString(randomBytes), nil
|
|
}
|
|
|
|
// generateAssumedRoleArn generates the ARN for an assumed role user
|
|
func GenerateAssumedRoleArn(roleArn, sessionName string) string {
|
|
// Convert role ARN to assumed role user ARN
|
|
// arn:aws:iam::role/RoleName -> arn:aws:sts::assumed-role/RoleName/SessionName
|
|
roleName := utils.ExtractRoleNameFromArn(roleArn)
|
|
if roleName == "" {
|
|
// This should not happen if validation is done properly upstream
|
|
return fmt.Sprintf("arn:aws:sts::assumed-role/INVALID-ARN/%s", sessionName)
|
|
}
|
|
return fmt.Sprintf("arn:aws:sts::assumed-role/%s/%s", roleName, sessionName)
|
|
}
|