Files
seaweedFS/weed/iam/sts/token_utils.go
Chris Lu ee3813787e feat(s3api): Implement S3 Policy Variables (#8039)
* feat: Add AWS IAM Policy Variables support to S3 API

Implements policy variables for dynamic access control in bucket policies.

Supported variables:
- aws:username - Extracted from principal ARN
- aws:userid - User identifier (same as username in SeaweedFS)
- aws:principaltype - IAMUser, IAMRole, or AssumedRole
- jwt:* - Any JWT claim (e.g., jwt:preferred_username, jwt:sub)

Key changes:
- Added PolicyVariableRegex to detect ${...} patterns
- Extended CompiledStatement with DynamicResourcePatterns, DynamicPrincipalPatterns, DynamicActionPatterns
- Added Claims field to PolicyEvaluationArgs for JWT claim access
- Implemented SubstituteVariables() for variable replacement from context and JWT claims
- Implemented extractPrincipalVariables() for ARN parsing
- Updated EvaluateConditions() to support variable substitution
- Comprehensive unit and integration tests

Resolves #8037

* feat: Add LDAP and PrincipalAccount variable support

Completes future enhancements for policy variables:

- Added ldap:* variable support for LDAP claims
  - ldap:username - LDAP username from claims
  - ldap:dn - LDAP distinguished name from claims
  - ldap:* - Any LDAP claim

- Added aws:PrincipalAccount extraction from ARN
  - Extracts account ID from principal ARN
  - Available as ${aws:PrincipalAccount} in policies

Updated SubstituteVariables() to check LDAP claims
Updated extractPrincipalVariables() to extract account ID
Added comprehensive tests for new variables

* feat(s3api): implement IAM policy variables core logic and optimization

* feat(s3api): integrate policy variables with S3 authentication and handlers

* test(s3api): add integration tests for policy variables

* cleanup: remove unused policy conversion files

* Add S3 policy variables integration tests and path support

- Add comprehensive integration tests for policy variables
- Test username isolation, JWT claims, LDAP claims
- Add support for IAM paths in principal ARN parsing
- Add tests for principals with paths

* Fix IAM Role principal variable extraction

IAM Roles should not have aws:userid or aws:PrincipalAccount
according to AWS behavior. Only IAM Users and Assumed Roles
should have these variables.

Fixes TestExtractPrincipalVariables test failures.

* Security fixes and bug fixes for S3 policy variables

SECURITY FIXES:
- Prevent X-SeaweedFS-Principal header spoofing by clearing internal
  headers at start of authentication (auth_credentials.go)
- Restrict policy variable substitution to safe allowlist to prevent
  client header injection (iam/policy/policy_engine.go)
- Add core policy validation before storing bucket policies

BUG FIXES:
- Remove unused sid variable in evaluateStatement
- Fix LDAP claim lookup to check both prefixed and unprefixed keys
- Add ValidatePolicy call in PutBucketPolicyHandler

These fixes prevent privilege escalation via header injection and
ensure only validated identity claims are used in policy evaluation.

* Additional security fixes and code cleanup

SECURITY FIXES:
- Fixed X-Forwarded-For spoofing by only trusting proxy headers from
  private/localhost IPs (s3_iam_middleware.go)
- Changed context key from "sourceIP" to "aws:SourceIp" for proper
  policy variable substitution

CODE IMPROVEMENTS:
- Kept aws:PrincipalAccount for IAM Roles to support condition evaluations
- Removed redundant STS principaltype override
- Removed unused service variable
- Cleaned up commented-out debug logging statements
- Updated tests to reflect new IAM Role behavior

These changes prevent IP spoofing attacks and ensure policy variables
work correctly with the safe allowlist.

* Add security documentation for ParseJWTToken

Added comprehensive security comments explaining that ParseJWTToken
is safe despite parsing without verification because:
- It's only used for routing to the correct verification method
- All code paths perform cryptographic verification before trusting claims
- OIDC tokens: validated via validateExternalOIDCToken
- STS tokens: validated via ValidateSessionToken

Enhanced function documentation with clear security warnings about
proper usage to prevent future misuse.

* Fix IP condition evaluation to use aws:SourceIp key

Fixed evaluateIPCondition in IAM policy engine to use "aws:SourceIp"
instead of "sourceIP" to match the updated extractRequestContext.

This fixes the failing IP-restricted role test where IP-based policy
conditions were not being evaluated correctly.

Updated all test cases to use the correct "aws:SourceIp" key.

* Address code review feedback: optimize and clarify

PERFORMANCE IMPROVEMENT:
- Optimized expandPolicyVariables to use regexp.ReplaceAllStringFunc
  for single-pass variable substitution instead of iterating through
  all safe variables. This improves performance from O(n*m) to O(m)
  where n is the number of safe variables and m is the pattern length.

CODE CLARITY:
- Added detailed comment explaining LDAP claim fallback mechanism
  (checks both prefixed and unprefixed keys for compatibility)
- Enhanced TODO comment for trusted proxy configuration with rationale
  and recommendations for supporting cloud load balancers, CDNs, and
  complex network topologies

All tests passing.

* Address Copilot code review feedback

BUG FIXES:
- Fixed type switch for int/int32/int64 - separated into individual cases
  since interface type switches only match the first type in multi-type cases
- Fixed grammatically incorrect error message in types.go

CODE QUALITY:
- Removed duplicate Resource/NotResource validation (already in ValidateStatement)
- Added comprehensive comment explaining isEnabled() logic and security implications
- Improved trusted proxy NOTE comment to be more concise while noting limitations

All tests passing.

* Fix test failures after extractSourceIP security changes

Updated tests to work with the security fix that only trusts
X-Forwarded-For/X-Real-IP headers from private IP addresses:

- Set RemoteAddr to 127.0.0.1 in tests to simulate trusted proxy
- Changed context key from "sourceIP" to "aws:SourceIp"
- Added test case for untrusted proxy (public RemoteAddr)
- Removed invalid ValidateStatement call (validation happens in ValidatePolicy)

All tests now passing.

* Address remaining Gemini code review feedback

CODE SAFETY:
- Deep clone Action field in CompileStatement to prevent potential data races
  if the original policy document is modified after compilation

TEST CLEANUP:
- Remove debug logging (fmt.Fprintf) from engine_notresource_test.go
- Remove unused imports in engine_notresource_test.go

All tests passing.

* Fix insecure JWT parsing in IAM auth flow

SECURITY FIX:
- Renamed ParseJWTToken to ParseUnverifiedJWTToken with explicit security warnings.
- Refactored AuthenticateJWT to use the trusted SessionInfo returned by ValidateSessionToken
  instead of relying on unverified claims from the initial parse.
- Refactored ValidatePresignedURLWithIAM to reuse the robust AuthenticateJWT logic, removing
  duplicated and insecure manual token parsing.

This ensures all identity information (Role, Principal, Subject) used for authorization
decisions is derived solely from cryptographically verified tokens.

* Security: Fix insecure JWT claim extraction in policy engine

- Refactored EvaluatePolicy to accept trusted claims from verified Identity instead of parsing unverified tokens
- Updated AuthenticateJWT to populate Claims in IAMIdentity from verified sources (SessionInfo/ExternalIdentity)
- Updated s3api_server and handlers to pass claims correctly
- Improved isPrivateIP to support IPv6 loopback, link-local, and ULA
- Fixed flaky distributed_session_consistency test with retry logic

* fix(iam): populate Subject in STSSessionInfo to ensure correct identity propagation

This fixes the TestS3IAMAuthentication/valid_jwt_token_authentication failure by ensuring the session subject (sub) is correctly mapped to the internal SessionInfo struct, allowing bucket ownership validation to succeed.

* Optimized isPrivateIP

* Create s3-policy-tests.yml

* fix tests

* fix tests

* tests(s3/iam): simplify policy to resource-based \ (step 1)

* tests(s3/iam): add explicit Deny NotResource for isolation (step 2)

* fixes

* policy: skip resource matching for STS trust policies to allow AssumeRole evaluation

* refactor: remove debug logging and hoist policy variables for performance

* test: fix TestS3IAMBucketPolicyIntegration cleanup to handle per-subtest object lifecycle

* test: fix bucket name generation to comply with S3 63-char limit

* test: skip TestS3IAMPolicyEnforcement until role setup is implemented

* test: use weed mini for simpler test server deployment

Replace 'weed server' with 'weed mini' for IAM tests to avoid port binding issues
and simplify the all-in-one server deployment. This improves test reliability
and execution time.

* security: prevent allocation overflow in policy evaluation

Add maxPoliciesForEvaluation constant to cap the number of policies evaluated
in a single request. This prevents potential integer overflow when allocating
slices for policy lists that may be influenced by untrusted input.

Changes:
- Add const maxPoliciesForEvaluation = 1024 to set an upper bound
- Validate len(policies) < maxPoliciesForEvaluation before appending bucket policy
- Use append() instead of make([]string, len+1) to avoid arithmetic overflow
- Apply fix to both IsActionAllowed policy evaluation paths
2026-01-16 11:12:28 -08:00

244 lines
7.7 KiB
Go

package sts
import (
"crypto/rand"
"crypto/sha256"
"encoding/base64"
"encoding/hex"
"encoding/json"
"fmt"
"time"
"github.com/golang-jwt/jwt/v5"
"github.com/seaweedfs/seaweedfs/weed/iam/utils"
)
// TokenGenerator handles token generation and validation
type TokenGenerator struct {
signingKey []byte
issuer string
}
// NewTokenGenerator creates a new token generator
func NewTokenGenerator(signingKey []byte, issuer string) *TokenGenerator {
return &TokenGenerator{
signingKey: signingKey,
issuer: issuer,
}
}
// GenerateSessionToken creates a signed JWT session token (legacy method for compatibility)
func (t *TokenGenerator) GenerateSessionToken(sessionId string, expiresAt time.Time) (string, error) {
claims := NewSTSSessionClaims(sessionId, t.issuer, expiresAt)
return t.GenerateJWTWithClaims(claims)
}
// GenerateJWTWithClaims creates a signed JWT token with comprehensive session claims
func (t *TokenGenerator) GenerateJWTWithClaims(claims *STSSessionClaims) (string, error) {
if claims == nil {
return "", fmt.Errorf("claims cannot be nil")
}
// Ensure issuer is set from token generator
if claims.Issuer == "" {
claims.Issuer = t.issuer
}
token := jwt.NewWithClaims(jwt.SigningMethodHS256, claims)
return token.SignedString(t.signingKey)
}
// ValidateSessionToken validates and extracts claims from a session token
func (t *TokenGenerator) ValidateSessionToken(tokenString string) (*SessionTokenClaims, error) {
token, err := jwt.Parse(tokenString, func(token *jwt.Token) (interface{}, error) {
if _, ok := token.Method.(*jwt.SigningMethodHMAC); !ok {
return nil, fmt.Errorf("unexpected signing method: %v", token.Header["alg"])
}
return t.signingKey, nil
})
if err != nil {
return nil, fmt.Errorf(ErrInvalidToken, err)
}
if !token.Valid {
return nil, fmt.Errorf(ErrTokenNotValid)
}
claims, ok := token.Claims.(jwt.MapClaims)
if !ok {
return nil, fmt.Errorf(ErrInvalidTokenClaims)
}
// Verify issuer
if iss, ok := claims[JWTClaimIssuer].(string); !ok || iss != t.issuer {
return nil, fmt.Errorf(ErrInvalidIssuer)
}
// Extract session ID
sessionId, ok := claims[JWTClaimSubject].(string)
if !ok {
return nil, fmt.Errorf(ErrMissingSessionID)
}
return &SessionTokenClaims{
SessionId: sessionId,
ExpiresAt: time.Unix(int64(claims[JWTClaimExpiration].(float64)), 0),
IssuedAt: time.Unix(int64(claims[JWTClaimIssuedAt].(float64)), 0),
}, nil
}
// ValidateJWTWithClaims validates and extracts comprehensive session claims from a JWT token
func (t *TokenGenerator) ValidateJWTWithClaims(tokenString string) (*STSSessionClaims, error) {
// 1. Parse into MapClaims to capture ALL claims including custom ones
token, err := jwt.Parse(tokenString, func(token *jwt.Token) (interface{}, error) {
if _, ok := token.Method.(*jwt.SigningMethodHMAC); !ok {
return nil, fmt.Errorf("unexpected signing method: %v", token.Header["alg"])
}
return t.signingKey, nil
})
if err != nil {
return nil, fmt.Errorf(ErrInvalidToken, err)
}
if !token.Valid {
return nil, fmt.Errorf(ErrTokenNotValid)
}
mapClaims, ok := token.Claims.(jwt.MapClaims)
if !ok {
return nil, fmt.Errorf(ErrInvalidTokenClaims)
}
// 2. Decode into STSSessionClaims using JSON round-trip to respect tags
jsonBytes, err := json.Marshal(mapClaims)
if err != nil {
return nil, fmt.Errorf("failed to marshal claims: %v", err)
}
claims := &STSSessionClaims{}
if err := json.Unmarshal(jsonBytes, claims); err != nil {
return nil, fmt.Errorf("failed to unmarshal claims: %v", err)
}
// 3. Ensure RequestContext contains all claims for policy evaluation
// This preserves custom claims (like jwt:preferred_username) that are not in the struct
if claims.RequestContext == nil {
claims.RequestContext = make(map[string]interface{})
}
for k, v := range mapClaims {
// Add valid claim values to RequestContext
// We don't overwrite existing RequestContext keys if they were explicitly set
if _, exists := claims.RequestContext[k]; !exists {
claims.RequestContext[k] = v
}
}
// Validate issuer
if claims.Issuer != t.issuer {
return nil, fmt.Errorf(ErrInvalidIssuer)
}
// Validate that required fields are present
if claims.SessionId == "" {
return nil, fmt.Errorf(ErrMissingSessionID)
}
// Additional validation using the claims' own validation method
if !claims.IsValid() {
return nil, fmt.Errorf(ErrTokenNotValid)
}
return claims, nil
}
// SessionTokenClaims represents parsed session token claims
type SessionTokenClaims struct {
SessionId string
ExpiresAt time.Time
IssuedAt time.Time
}
// CredentialGenerator generates AWS-compatible temporary credentials
type CredentialGenerator struct{}
// NewCredentialGenerator creates a new credential generator
func NewCredentialGenerator() *CredentialGenerator {
return &CredentialGenerator{}
}
// GenerateTemporaryCredentials creates temporary AWS credentials
func (c *CredentialGenerator) GenerateTemporaryCredentials(sessionId string, expiration time.Time) (*Credentials, error) {
accessKeyId, err := c.generateAccessKeyId(sessionId)
if err != nil {
return nil, fmt.Errorf("failed to generate access key ID: %w", err)
}
secretAccessKey, err := c.generateSecretAccessKey(sessionId)
if err != nil {
return nil, fmt.Errorf("failed to generate secret access key: %w", err)
}
sessionToken, err := c.generateSessionTokenId(sessionId)
if err != nil {
return nil, fmt.Errorf("failed to generate session token: %w", err)
}
return &Credentials{
AccessKeyId: accessKeyId,
SecretAccessKey: secretAccessKey,
SessionToken: sessionToken,
Expiration: expiration,
}, nil
}
// generateAccessKeyId generates an AWS-style access key ID
func (c *CredentialGenerator) generateAccessKeyId(sessionId string) (string, error) {
// Create a deterministic but unique access key ID based on session
hash := sha256.Sum256([]byte("access-key:" + sessionId))
return "AKIA" + hex.EncodeToString(hash[:8]), nil // AWS format: AKIA + 16 chars
}
// generateSecretAccessKey generates a deterministic secret access key based on sessionId
// This ensures the same secret key is regenerated from the JWT claims during signature verification
func (c *CredentialGenerator) generateSecretAccessKey(sessionId string) (string, error) {
// Create deterministic secret key based on session ID (not random!)
// This is critical for STS because:
// 1. AssumeRoleWithWebIdentity generates the secret key once
// 2. During signature verification, ToSessionInfo() regenerates credentials from JWT
// 3. Both must generate the same secret key for signature verification to succeed
hash := sha256.Sum256([]byte("secret-key:" + sessionId))
return base64.StdEncoding.EncodeToString(hash[:]), nil
}
// generateSessionTokenId generates a session token identifier
func (c *CredentialGenerator) generateSessionTokenId(sessionId string) (string, error) {
// Create session token with session ID embedded
hash := sha256.Sum256([]byte("session-token:" + sessionId))
return "ST" + hex.EncodeToString(hash[:16]), nil // Custom format
}
// generateSessionId generates a unique session ID
func GenerateSessionId() (string, error) {
randomBytes := make([]byte, 16)
_, err := rand.Read(randomBytes)
if err != nil {
return "", err
}
return hex.EncodeToString(randomBytes), nil
}
// generateAssumedRoleArn generates the ARN for an assumed role user
func GenerateAssumedRoleArn(roleArn, sessionName string) string {
// Convert role ARN to assumed role user ARN
// arn:aws:iam::role/RoleName -> arn:aws:sts::assumed-role/RoleName/SessionName
roleName := utils.ExtractRoleNameFromArn(roleArn)
if roleName == "" {
// This should not happen if validation is done properly upstream
return fmt.Sprintf("arn:aws:sts::assumed-role/INVALID-ARN/%s", sessionName)
}
return fmt.Sprintf("arn:aws:sts::assumed-role/%s/%s", roleName, sessionName)
}