- Remove mandatory ACTION_ADMIN at the router level
- Enforce granular permissions in bucket and namespace handlers
- Prioritize AccountID in ExtractPrincipalFromContext for ARN matching
- Distinguish between 404 (NoSuchBucket) and 500 (InternalError) in metadata lookups
- Clean up unused imports in s3api_tables.go
- Update generateTableARN to match AWS S3 Tables specification
- Move defer r.Body.Close() to follow standard Go patterns
- Remove unused generateNamespaceARN helper
- Fix InclusiveStartFrom logic to ensure exclusive start on continued pages
- Prevent duplicates in bucket, namespace, and table listings
- Fail fast on listing errors during bucket and namespace deletion
- Stop swallowing errors in handleListTables and return proper HTTP error responses
- Add multi-segment namespace support to ARN parsing
- Refactor permission checking to use map lookup
- Wrap lookup errors with ErrNotFound in filer operations
- Standardize splitPath to use path package
- Add permissions.go with permission definitions and checks
- Define permissions for all 21 S3 Tables operations
- Add permission checking helper functions
- Add getPrincipalFromRequest to extract caller identity
- Implement access control in CreateTableBucket, GetTableBucket, DeleteTableBucket
- Return 403 Forbidden for unauthorized operations
- Only bucket owner can perform operations (extensible for future policies)
- Add AuthError type for authorization failures
- Replace O(n) slice iteration with O(1) map lookup
- Move s3TablesActionsMap to package level
- Avoid recreating the map on every function call
- Improves performance for request validation
- Specifically check for 'not found' errors instead of catching all errors
- Return empty list only when directory doesn't exist
- Propagate other errors (network, permission) with context
- Prevents masking real errors
- Add S3 Tables route registration in s3api_server.go registerRouter method
- Enable S3 Tables API operations to be routed through S3 API server
- Routes handled by s3api_tables.go integration layer
- Minimal changes to existing S3 API structure
- Create s3api_tables.go to integrate S3 Tables with S3 API server
- Implement S3 Tables route matcher for X-Amz-Target header
- Register S3 Tables routes with API router
- Provide gRPC filer client interface for S3 Tables handlers
- All S3 Tables operations accessible via S3 API endpoint
- Reduce handler.go from 370 to 195 lines (47% reduction)
- Remove duplicate ARN parsing and path helper functions
- Remove filer operation methods moved to filer_ops.go
- Remove metadata structure definitions moved to utils.go
- Keep handler focused on request routing and response formatting
- Maintains all functionality with improved code organization
- Create bucket_create.go for CreateTableBucket operation
- Create bucket_get_list_delete.go for Get, List, Delete operations
- Related operations grouped for better maintainability
- Each file has a single, clear responsibility
- Improves code clarity and makes it easier to test
- Move ARN parsing, path helpers, and metadata structures to utils.go
- Extract all extended attribute and filer operations to filer_ops.go
- Reduces code duplication and improves modularity
- Improves code organization and maintainability
* Implement IAM propagation to S3 servers
- Add PropagatingCredentialStore to propagate IAM changes to S3 servers via gRPC
- Add Policy management RPCs to S3 proto and S3ApiServer
- Update CredentialManager to use PropagatingCredentialStore when MasterClient is available
- Wire FilerServer to enable propagation
* Implement parallel IAM propagation and fix S3 cluster registration
- Parallelized IAM change propagation with 10s timeout.
- Refined context usage in PropagatingCredentialStore.
- Added S3Type support to cluster node management.
- Enabled S3 servers to register with gRPC address to the master.
- Ensured IAM configuration reload after policy updates via gRPC.
* Optimize IAM propagation with direct in-memory cache updates
* Secure IAM propagation: Use metadata to skip persistence only on propagation
* pb: refactor IAM and S3 services for unidirectional IAM propagation
- Move SeaweedS3IamCache service from iam.proto to s3.proto.
- Remove legacy IAM management RPCs and empty SeaweedS3 service from s3.proto.
- Enforce that S3 servers only use the synchronization interface.
* pb: regenerate Go code for IAM and S3 services
Updated generated code following the proto refactoring of IAM synchronization services.
* s3api: implement read-only mode for Embedded IAM API
- Add readOnly flag to EmbeddedIamApi to reject write operations via HTTP.
- Enable read-only mode by default in S3ApiServer.
- Handle AccessDenied error in writeIamErrorResponse.
- Embed SeaweedS3IamCacheServer in S3ApiServer.
* credential: refactor PropagatingCredentialStore for unidirectional IAM flow
- Update to use s3_pb.SeaweedS3IamCacheClient for propagation to S3 servers.
- Propagate full Identity object via PutIdentity for consistency.
- Remove redundant propagation of specific user/account/policy management RPCs.
- Add timeout context for propagation calls.
* s3api: implement SeaweedS3IamCacheServer for unidirectional sync
- Update S3ApiServer to implement the cache synchronization gRPC interface.
- Methods (PutIdentity, RemoveIdentity, etc.) now perform direct in-memory cache updates.
- Register SeaweedS3IamCacheServer in command/s3.go.
- Remove registration for the legacy and now empty SeaweedS3 service.
* s3api: update tests for read-only IAM and propagation
- Added TestEmbeddedIamReadOnly to verify rejection of write operations in read-only mode.
- Update test setup to pass readOnly=false to NewEmbeddedIamApi in routing tests.
- Updated EmbeddedIamApiForTest helper with read-only checks matching production behavior.
* s3api: add back temporary debug logs for IAM updates
Log IAM updates received via:
- gRPC propagation (PutIdentity, PutPolicy, etc.)
- Metadata configuration reloads (LoadS3ApiConfigurationFromCredentialManager)
- Core identity management (UpsertIdentity, RemoveIdentity)
* IAM: finalize propagation fix with reduced logging and clarified architecture
* Allow configuring IAM read-only mode for S3 server integration tests
* s3api: add defensive validation to UpsertIdentity
* s3api: fix log message to reference correct IAM read-only flag
* test/s3/iam: ensure WaitForS3Service checks for IAM write permissions
* test: enable writable IAM in Makefile for integration tests
* IAM: add GetPolicy/ListPolicies RPCs to s3.proto
* S3: add GetBucketPolicy and ListBucketPolicies helpers
* S3: support storing generic IAM policies in IdentityAccessManagement
* S3: implement IAM policy RPCs using IdentityAccessManagement
* IAM: fix stale user identity on rename propagation
Recent changes in the S3 unified copy strategy were constructing source
and destination paths without the necessary BucketsPath prefix (typically
/buckets). When these paths reached the Filer for volume assignment, it
failed to resolve the correct collection and storage rules, defaulting to
a disk type with no available capacity.
This fix ensures all relevant paths in S3 copy handlers include the
correct BucketsPath prefix for proper collection resolution.
Fixes replication issue with Harbor upload via S3 API.
* Update IAM and S3 protobuf definitions for explicit IAM gRPC APIs
* Refactor s3api: Extract generic ExecuteAction method for IAM operations
* Implement explicit IAM gRPC APIs in S3 server
* iam: remove deprecated GetConfiguration and PutConfiguration RPCs
* iamapi: refactor handlers to use CredentialManager directly
* s3api: refactor embedded IAM to use CredentialManager directly
* server: remove deprecated configuration gRPC handlers
* credential/grpc: refactor configuration calls to return error
* shell: update s3.configure to list users instead of full config
* s3api: fix CreateServiceAccount gRPC handler to map required fields
* s3api: fix UpdateServiceAccount gRPC handler to map fields and safe status
* s3api: enforce UserName in embedded IAM ListAccessKeys
* test: fix test_config.json structure to match proto definition
* Revert "credential/grpc: refactor configuration calls to return error"
This reverts commit cde707dd8b88c7d1bd730271518542eceb5ed069.
* Revert "server: remove deprecated configuration gRPC handlers"
This reverts commit 7307e205a083c8315cf84ddc2614b3e50eda2e33.
* Revert "s3api: enforce UserName in embedded IAM ListAccessKeys"
This reverts commit adf727ba52b4f3ffb911f0d0df85db858412ff83.
* Revert "s3api: fix UpdateServiceAccount gRPC handler to map fields and safe status"
This reverts commit 6a4be3314d43b6c8fda8d5e0558e83e87a19df3f.
* Revert "s3api: fix CreateServiceAccount gRPC handler to map required fields"
This reverts commit 9bb4425f07fbad38fb68d33e5c0aa573d8912a37.
* Revert "shell: update s3.configure to list users instead of full config"
This reverts commit f3304ead537b3e6be03d46df4cb55983ab931726.
* Revert "s3api: refactor embedded IAM to use CredentialManager directly"
This reverts commit 9012f27af82d11f0e824877712a5ae2505a65f86.
* Revert "iamapi: refactor handlers to use CredentialManager directly"
This reverts commit 3a148212236576b0a3aa4d991c2abb014fb46091.
* Revert "iam: remove deprecated GetConfiguration and PutConfiguration RPCs"
This reverts commit e16e08aa0099699338d3155bc7428e1051ce0a6a.
* s3api: address IAM code review comments (error handling, logging, gRPC response mapping)
* s3api: add robustness to startup by retrying KEK and IAM config loading from Filer
* s3api: address IAM gRPC code review comments (safety, validation, status logic)
* fix return
* Refactor Admin UI to use unified IAM storage and add MultipleFileStore
* Address PR feedback: fix renames, error handling, and sync logic in FilerMultipleStore
* Address refined PR feedback: safe rename order, rollback logic, and structural sync refinement
* Optimize LoadConfiguration: use streaming callback for memory efficiency
* Refactor UpdateUser: log rollback failures during rename
* Implement PolicyManager for FilerMultipleStore
* include the filer_multiple backend configuration
* Implement cross-S3 synchronization and proper shutdown for all IAM backends
* Extract Admin UI refactoring to a separate PR
* Add AWS IAM integration tests and refactor admin authorization
- Added AWS IAM management integration tests (User, AccessKey, Policy)
- Updated test framework to support IAM client creation with JWT/OIDC
- Refactored s3api authorization to be policy-driven for IAM actions
- Removed hardcoded role name checks for admin privileges
- Added new tests to GitHub Actions basic test matrix
* test(s3/iam): add UpdateUser and UpdateAccessKey tests and fix nil pointer dereference
* feat(s3api): add DeletePolicy and update tests with cleanup logic
* test(s3/iam): use t.Cleanup for managed policy deletion in CreatePolicy test
* fix: IAM authentication with AWS Signature V4 and environment credentials
Three key fixes for authenticated IAM requests to work:
1. Fix request body consumption before signature verification
- iamMatcher was calling r.ParseForm() which consumed POST body
- This broke AWS Signature V4 verification on subsequent reads
- Now only check query string in matcher, preserving body for verification
- File: weed/s3api/s3api_server.go
2. Preserve environment variable credentials across config reloads
- After IAM mutations, config reload overwrote env var credentials
- Extract env var loading into loadEnvironmentVariableCredentials()
- Call after every config reload to persist credentials
- File: weed/s3api/auth_credentials.go
3. Add authenticated IAM tests and test infrastructure
- New TestIAMAuthenticated suite with AWS SDK + Signature V4
- Dynamic port allocation for independent test execution
- Flag reset to prevent state leakage between tests
- CI workflow to run S3 and IAM tests separately
- Files: test/s3/example/*, .github/workflows/s3-example-integration-tests.yml
All tests pass:
- TestIAMCreateUser (unauthenticated)
- TestIAMAuthenticated (with AWS Signature V4)
- S3 integration tests
* fmt
* chore: rename test/s3/example to test/s3/normal
* simplify: CI runs all integration tests in single job
* Update s3-example-integration-tests.yml
* ci: run each test group separately to avoid raft registry conflicts
* fix: S3 listing NextMarker missing intermediate directory component
When listing with nested prefixes like "character/member/", the NextMarker
was incorrectly constructed as "character/res024/" instead of
"character/member/res024/", causing continuation requests to fail.
Root cause: The code at line 331 was constructing NextMarker as:
nextMarker = requestDir + "/" + nextMarker
This worked when nextMarker already contained the full relative path,
but failed when it was just the entry name from the innermost recursion.
Fix: Include the prefix component when constructing NextMarker:
if prefix != "" {
nextMarker = requestDir + "/" + prefix + "/" + nextMarker
}
This ensures the full path is always constructed correctly for both:
- CommonPrefix entries (directories)
- Regular entries (files)
Also includes fix for cursor.prefixEndsOnDelimiter state leak that was
causing sibling directories to be incorrectly listed.
* test: add regression tests for NextMarker construction
Add comprehensive unit tests to verify NextMarker is correctly constructed
with nested prefixes. Tests cover:
- Regular entries with nested prefix (character/member/res024)
- CommonPrefix entries (directories)
- Edge cases (no requestDir, no prefix, deeply nested)
These tests ensure the fix prevents regression of the bug where
NextMarker was missing intermediate directory components.
* Fix: Populate Claims from STS session RequestContext for policy variable substitution
When using STS temporary credentials (from AssumeRoleWithWebIdentity) with
AWS Signature V4 authentication, JWT claims like preferred_username were
not available for bucket policy variable substitution (e.g., ${jwt:preferred_username}).
Root Cause:
- STS session tokens store user claims in the req_ctx field (added in PR #8079)
- validateSTSSessionToken() created Identity but didn't populate Claims field
- authorizeWithIAM() created IAMIdentity but didn't copy Claims
- Policy engine couldn't resolve ${jwt:*} variables without claims
Changes:
1. auth_signature_v4.go: Extract claims from sessionInfo.RequestContext
and populate Identity.Claims in validateSTSSessionToken()
2. auth_credentials.go: Copy Claims when creating IAMIdentity in
authorizeWithIAM()
3. auth_sts_identity_test.go: Add TestSTSIdentityClaimsPopulation to
verify claims are properly populated from RequestContext
This enables bucket policies with JWT claim variables to work correctly
with STS temporary credentials obtained via AssumeRoleWithWebIdentity.
Fixes#8037
* Refactor: Idiomatic map population for STS claims
* Fix S3 conditional writes with versioning (Issue #8073)
Refactors conditional header checks to properly resolve the latest object version when versioning is enabled. This prevents incorrect validation against non-versioned root objects.
* Add integration test for S3 conditional writes with versioning (Issue #8073)
* Refactor: Propagate internal errors in conditional header checks
- Make resolveObjectEntry return errors from isVersioningConfigured
- Update checkConditionalHeaders checks to return 500 on internal resolve errors
* Refactor: Stricter error handling and test assertions
- Propagate internal errors in checkConditionalHeaders*WithGetter functions
- Enforce strict 412 PreconditionFailed check in integration test
* Perf: Add early return for conditional headers + safety improvements
- Add fast path to skip resolveObjectEntry when no conditional headers present
- Avoids expensive getLatestObjectVersion retries in common case
- Add nil checks before dereferencing pointers in integration test
- Fix grammar in test comments
- Remove duplicate comment in resolveObjectEntry
* Refactor: Use errors.Is for robust ErrNotFound checking
- Update checkConditionalHeaders* to use errors.Is(err, filer_pb.ErrNotFound)
- Update resolveObjectEntry to use errors.Is for wrapped error compatibility
- Remove duplicate comment lines in s3api handlers
* Perf: Optimize resolveObjectEntry for conditional checks
- Refactor getLatestObjectVersion to doGetLatestObjectVersion supporting variable retries
- Use 1-retry path in resolveObjectEntry to avoid exponential backoff latency
* Test: Enhance integration test with content verification
- Verify actual object content equals expected content after successful conditional write
- Add missing io and errors imports to test file
* Refactor: Final refinements based on feedback
- Optimize header validation by passing parsed headers to avoid redundant parsing
- Simplify integration test assertions using require.Error and assert.True
- Fix build errors in s3api handler and test imports
* Test: Use smithy.APIError for robust error code checking
- Replace string-based error checking with structured API error
- Add smithy-go import for AWS SDK v2 error handling
* Test: Use types.PreconditionFailed and handle io.ReadAll error
- Replace smithy.APIError with more specific types.PreconditionFailed
- Add proper error handling for io.ReadAll in content verification
* Refactor: Use combined error checking and add nil guards
- Use smithy.APIError with ErrorCode() for robust error checking
- Add nil guards for entry.Attributes before accessing Mtime
- Prevents potential panics when Attributes is uninitialized
* fix: Refactor CORS middleware to consistently apply the `Vary: Origin` header when a configuration exists and streamline request processing logic.
* fix: Add Vary: Origin header to CORS OPTIONS responses and refactor request handling for clarity and correctness.
* fix: update CORS middleware tests to correctly parse and check for 'Origin' in Vary header.
* refactor: extract `hasVaryOrigin` helper function to simplify Vary header checks in tests.
* test: Remove `Vary: Origin` header from CORS test expectations.
* refactor: consolidate CORS request handling into a new `processCORS` method using a `next` callback.
Fix S3 CORS for non-existent buckets
Enable fallback to global CORS configuration when a bucket is not found (s3err.ErrNoSuchBucket). This ensures consistent CORS behavior and prevents information disclosure.
Fixes#8065
Problem:
- CORS headers were only applied after checking bucket existence
- Non-existent buckets returned responses without CORS headers
- This caused CORS preflight failures and information disclosure vulnerability
- Unauthenticated users could infer bucket existence from CORS header presence
Solution:
- Moved CORS evaluation before bucket existence check in middleware
- CORS headers now applied consistently regardless of bucket existence
- Preflight requests succeed for non-existent buckets (matching AWS S3)
- Actual requests still return NoSuchBucket error but with CORS headers
Changes:
- Modified Handler() and HandleOptionsRequest() in middleware.go
- Added comprehensive test suite for non-existent bucket scenarios
- All 39 tests passing (31 existing + 8 new)
Security Impact:
- Prevents information disclosure about bucket existence
- Bucket existence cannot be inferred from CORS header presence/absence
AWS S3 Compatibility:
- Improved compatibility with AWS S3 CORS behavior
- Preflight requests now succeed for non-existent buckets
Fix: Propagate OIDC claims to IAM identity for dynamic policy variables
Fixes#8037. Ensures additional OIDC claims (like preferred_username) are preserved in ExternalIdentity attributes and propagated to IAM tokens, enabling substitution in dynamic policies.
* filer: auto clean empty s3 implicit folders
Explicitly tag implicitly created S3 folders (parent directories from object uploads) with 'Seaweed-X-Amz-Implicit-Dir'.
Update EmptyFolderCleaner to check for this attribute and cache the result efficiently.
* filer: correctly handle nil attributes in empty folder cleaner cache
* filer: refine implicit tagging logic
Prevent tagging buckets as implicit directories. Reduce code duplication.
* filer: safeguard GetEntryAttributes against nil entry and not found error
* filer: move ErrNotFound handling to EmptyFolderCleaner
* filer: add comment to explain level > 3 check for implicit directories
* feat: Add AWS IAM Policy Variables support to S3 API
Implements policy variables for dynamic access control in bucket policies.
Supported variables:
- aws:username - Extracted from principal ARN
- aws:userid - User identifier (same as username in SeaweedFS)
- aws:principaltype - IAMUser, IAMRole, or AssumedRole
- jwt:* - Any JWT claim (e.g., jwt:preferred_username, jwt:sub)
Key changes:
- Added PolicyVariableRegex to detect ${...} patterns
- Extended CompiledStatement with DynamicResourcePatterns, DynamicPrincipalPatterns, DynamicActionPatterns
- Added Claims field to PolicyEvaluationArgs for JWT claim access
- Implemented SubstituteVariables() for variable replacement from context and JWT claims
- Implemented extractPrincipalVariables() for ARN parsing
- Updated EvaluateConditions() to support variable substitution
- Comprehensive unit and integration tests
Resolves#8037
* feat: Add LDAP and PrincipalAccount variable support
Completes future enhancements for policy variables:
- Added ldap:* variable support for LDAP claims
- ldap:username - LDAP username from claims
- ldap:dn - LDAP distinguished name from claims
- ldap:* - Any LDAP claim
- Added aws:PrincipalAccount extraction from ARN
- Extracts account ID from principal ARN
- Available as ${aws:PrincipalAccount} in policies
Updated SubstituteVariables() to check LDAP claims
Updated extractPrincipalVariables() to extract account ID
Added comprehensive tests for new variables
* feat(s3api): implement IAM policy variables core logic and optimization
* feat(s3api): integrate policy variables with S3 authentication and handlers
* test(s3api): add integration tests for policy variables
* cleanup: remove unused policy conversion files
* Add S3 policy variables integration tests and path support
- Add comprehensive integration tests for policy variables
- Test username isolation, JWT claims, LDAP claims
- Add support for IAM paths in principal ARN parsing
- Add tests for principals with paths
* Fix IAM Role principal variable extraction
IAM Roles should not have aws:userid or aws:PrincipalAccount
according to AWS behavior. Only IAM Users and Assumed Roles
should have these variables.
Fixes TestExtractPrincipalVariables test failures.
* Security fixes and bug fixes for S3 policy variables
SECURITY FIXES:
- Prevent X-SeaweedFS-Principal header spoofing by clearing internal
headers at start of authentication (auth_credentials.go)
- Restrict policy variable substitution to safe allowlist to prevent
client header injection (iam/policy/policy_engine.go)
- Add core policy validation before storing bucket policies
BUG FIXES:
- Remove unused sid variable in evaluateStatement
- Fix LDAP claim lookup to check both prefixed and unprefixed keys
- Add ValidatePolicy call in PutBucketPolicyHandler
These fixes prevent privilege escalation via header injection and
ensure only validated identity claims are used in policy evaluation.
* Additional security fixes and code cleanup
SECURITY FIXES:
- Fixed X-Forwarded-For spoofing by only trusting proxy headers from
private/localhost IPs (s3_iam_middleware.go)
- Changed context key from "sourceIP" to "aws:SourceIp" for proper
policy variable substitution
CODE IMPROVEMENTS:
- Kept aws:PrincipalAccount for IAM Roles to support condition evaluations
- Removed redundant STS principaltype override
- Removed unused service variable
- Cleaned up commented-out debug logging statements
- Updated tests to reflect new IAM Role behavior
These changes prevent IP spoofing attacks and ensure policy variables
work correctly with the safe allowlist.
* Add security documentation for ParseJWTToken
Added comprehensive security comments explaining that ParseJWTToken
is safe despite parsing without verification because:
- It's only used for routing to the correct verification method
- All code paths perform cryptographic verification before trusting claims
- OIDC tokens: validated via validateExternalOIDCToken
- STS tokens: validated via ValidateSessionToken
Enhanced function documentation with clear security warnings about
proper usage to prevent future misuse.
* Fix IP condition evaluation to use aws:SourceIp key
Fixed evaluateIPCondition in IAM policy engine to use "aws:SourceIp"
instead of "sourceIP" to match the updated extractRequestContext.
This fixes the failing IP-restricted role test where IP-based policy
conditions were not being evaluated correctly.
Updated all test cases to use the correct "aws:SourceIp" key.
* Address code review feedback: optimize and clarify
PERFORMANCE IMPROVEMENT:
- Optimized expandPolicyVariables to use regexp.ReplaceAllStringFunc
for single-pass variable substitution instead of iterating through
all safe variables. This improves performance from O(n*m) to O(m)
where n is the number of safe variables and m is the pattern length.
CODE CLARITY:
- Added detailed comment explaining LDAP claim fallback mechanism
(checks both prefixed and unprefixed keys for compatibility)
- Enhanced TODO comment for trusted proxy configuration with rationale
and recommendations for supporting cloud load balancers, CDNs, and
complex network topologies
All tests passing.
* Address Copilot code review feedback
BUG FIXES:
- Fixed type switch for int/int32/int64 - separated into individual cases
since interface type switches only match the first type in multi-type cases
- Fixed grammatically incorrect error message in types.go
CODE QUALITY:
- Removed duplicate Resource/NotResource validation (already in ValidateStatement)
- Added comprehensive comment explaining isEnabled() logic and security implications
- Improved trusted proxy NOTE comment to be more concise while noting limitations
All tests passing.
* Fix test failures after extractSourceIP security changes
Updated tests to work with the security fix that only trusts
X-Forwarded-For/X-Real-IP headers from private IP addresses:
- Set RemoteAddr to 127.0.0.1 in tests to simulate trusted proxy
- Changed context key from "sourceIP" to "aws:SourceIp"
- Added test case for untrusted proxy (public RemoteAddr)
- Removed invalid ValidateStatement call (validation happens in ValidatePolicy)
All tests now passing.
* Address remaining Gemini code review feedback
CODE SAFETY:
- Deep clone Action field in CompileStatement to prevent potential data races
if the original policy document is modified after compilation
TEST CLEANUP:
- Remove debug logging (fmt.Fprintf) from engine_notresource_test.go
- Remove unused imports in engine_notresource_test.go
All tests passing.
* Fix insecure JWT parsing in IAM auth flow
SECURITY FIX:
- Renamed ParseJWTToken to ParseUnverifiedJWTToken with explicit security warnings.
- Refactored AuthenticateJWT to use the trusted SessionInfo returned by ValidateSessionToken
instead of relying on unverified claims from the initial parse.
- Refactored ValidatePresignedURLWithIAM to reuse the robust AuthenticateJWT logic, removing
duplicated and insecure manual token parsing.
This ensures all identity information (Role, Principal, Subject) used for authorization
decisions is derived solely from cryptographically verified tokens.
* Security: Fix insecure JWT claim extraction in policy engine
- Refactored EvaluatePolicy to accept trusted claims from verified Identity instead of parsing unverified tokens
- Updated AuthenticateJWT to populate Claims in IAMIdentity from verified sources (SessionInfo/ExternalIdentity)
- Updated s3api_server and handlers to pass claims correctly
- Improved isPrivateIP to support IPv6 loopback, link-local, and ULA
- Fixed flaky distributed_session_consistency test with retry logic
* fix(iam): populate Subject in STSSessionInfo to ensure correct identity propagation
This fixes the TestS3IAMAuthentication/valid_jwt_token_authentication failure by ensuring the session subject (sub) is correctly mapped to the internal SessionInfo struct, allowing bucket ownership validation to succeed.
* Optimized isPrivateIP
* Create s3-policy-tests.yml
* fix tests
* fix tests
* tests(s3/iam): simplify policy to resource-based \ (step 1)
* tests(s3/iam): add explicit Deny NotResource for isolation (step 2)
* fixes
* policy: skip resource matching for STS trust policies to allow AssumeRole evaluation
* refactor: remove debug logging and hoist policy variables for performance
* test: fix TestS3IAMBucketPolicyIntegration cleanup to handle per-subtest object lifecycle
* test: fix bucket name generation to comply with S3 63-char limit
* test: skip TestS3IAMPolicyEnforcement until role setup is implemented
* test: use weed mini for simpler test server deployment
Replace 'weed server' with 'weed mini' for IAM tests to avoid port binding issues
and simplify the all-in-one server deployment. This improves test reliability
and execution time.
* security: prevent allocation overflow in policy evaluation
Add maxPoliciesForEvaluation constant to cap the number of policies evaluated
in a single request. This prevents potential integer overflow when allocating
slices for policy lists that may be influenced by untrusted input.
Changes:
- Add const maxPoliciesForEvaluation = 1024 to set an upper bound
- Validate len(policies) < maxPoliciesForEvaluation before appending bucket policy
- Use append() instead of make([]string, len+1) to avoid arithmetic overflow
- Apply fix to both IsActionAllowed policy evaluation paths
* use "s" flag of regexp to let . match \n
the partten "/{object:.+}" cause the mux failed to match URI of object
with new line char, and the request fall thru into bucket handlers.
* refactor
---------
Co-authored-by: Chris Lu <chris.lu@gmail.com>
