Commit Graph

12725 Commits

Author SHA1 Message Date
Chris Lu
01da26fbcb ci: Pin GitHub Actions to commit SHAs for s3-tables-tests
Update all action refs to use pinned commit SHAs instead of floating tags:
- actions/checkout: @v6 → @8e8c483 (v4)
- actions/setup-go: @v6 → @0c52d54 (v5)
- actions/upload-artifact: @v6 → @65d8626 (v4)

Pinned SHAs improve reproducibility and reduce supply chain risk by
preventing accidental or malicious changes in action releases. Aligns
with repository conventions used in other workflows (e.g., go.yml).
2026-01-28 17:40:40 -08:00
Chris Lu
2c45b69775 s3tables: Fix remaining policy error handling in namespace and bucket handlers
Replace silent error swallowing (err == nil) with proper error distinction
for bucket policy reads. Now properly checks ErrAttributeNotFound and
propagates other errors as internal server errors.

Fixed 5 locations:
- handleCreateNamespace (policy fetch)
- handleDeleteNamespace (policy fetch)
- handleListNamespaces (policy fetch)
- handleGetNamespace (policy fetch)
- handleGetTableBucket (policy fetch)

This prevents masking of filer issues when policies cannot be read due
to I/O errors or other transient failures.
2026-01-28 17:39:44 -08:00
Chris Lu
b7bba7e7dc s3tables: Generate ARNs using resource owner account ID
Change ARN generation to use resource OwnerAccountID instead of caller
identity (h.getAccountID(r)). This ensures ARNs are stable and consistent
regardless of which principal accesses the resource.

Updated generateTableBucketARN and generateTableARN function signatures
to accept ownerAccountID parameter. All call sites updated to pass the
resource owner's account ID from metadata.

This prevents ARN inconsistency issues when multiple principals have
access to the same resource via policies.
2026-01-28 17:38:22 -08:00
Chris Lu
e7b2869aa9 s3tables: Use policy framework for GetTable authorization
Replace strict ownership check with policy-based authorization in GetTable.
Now checks both table and bucket policies for GetTable permission, allowing
authorized non-owners to read table metadata.

Authorization logic:
- Table policy grants GetTable → allowed
- Bucket policy grants GetTable → allowed
- Otherwise → 404 NotFound (no access disclosed)

Maintains security through policy evaluation while enabling read delegation.
2026-01-28 17:37:12 -08:00
Chris Lu
bea0f8eda0 s3tables: Use policy framework for table creation authorization
Replace strict ownership check in CreateTable with policy-based authorization.
Now checks both namespace and bucket policies for CreateTable permission,
allowing delegation via resource policies while still respecting owner bypass.

Authorization logic:
- Namespace policy grants CreateTable → allowed
- Bucket policy grants CreateTable → allowed
- Otherwise → denied (even if same owner)

This enables cross-principal table creation via policies while maintaining
security through explicit allow/deny semantics.
2026-01-28 17:36:53 -08:00
Chris Lu
cf5043a9f9 s3tables: Normalize action names to include service prefix
Add automatic normalization of operations to full IAM-style action names
(e.g., 's3tables:CreateTableBucket') in CheckPermission(). This ensures
policy statements using prefixed actions (s3tables:*) correctly match
operations evaluated by permission helpers.

Also fixes incorrect r.Context() passed to GetIdentityNameFromContext
which expects *http.Request. Now passes r directly.
2026-01-28 17:36:16 -08:00
Chris Lu
ee468749bd Update weed/s3api/s3tables/handler.go
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-01-28 17:12:18 -08:00
Chris Lu
08bd1e2563 s3tables: Pre-validate namespace to return 400 instead of 500
Move validateNamespace call outside of filerClient.WithFilerClient closure
so that validation errors return HTTP 400 (InvalidRequest) instead of 500
(InternalError).

Before: Validation error inside closure → treated as internal error → 500
After: Validation error before closure → handled as bad request → 400

This provides correct error semantics: namespace validation is an input
validation issue, not a server error.
2026-01-28 17:03:04 -08:00
Chris Lu
8eee6b2a0e s3tables: Fix bucket policy error handling in permission checks
Replace error-swallowing pattern where all errors from getExtendedAttribute
were ignored for bucket policy reads. Now properly distinguish between:

- ErrAttributeNotFound: Policy not found is expected; continue with empty policy
- Other errors: Return internal server error and stop processing

Applied fix to all bucket policy reads in:
- handleDeleteTableBucketPolicy (line 220)
- handleTagResource (line 313)
- handleUntagResource (line 405)
- handleListTagsForResource (line 488)
- And additional occurrences in closures

This prevents silent failures and ensures policy-related errors are surfaced
to callers rather than being silently ignored.
2026-01-28 17:02:59 -08:00
Chris Lu
fe856928c4 s3tables: Add t field to TestCluster for logging
Add *testing.T field to TestCluster struct and initialize it in
startMiniCluster. This allows Stop() to properly log warnings when
cluster shutdown times out. Includes the t field in the test cluster
initialization and restores the logging statement in Stop().
2026-01-28 17:02:53 -08:00
Chris Lu
6658a655f6 clean up 2026-01-28 17:00:42 -08:00
Chris Lu
c5eadadf5a s3tables: Fix vet error - remove undefined c.t reference in Stop()
The TestCluster.Stop() method doesn't have access to testing.T object.
Remove the log statement and keep the timeout handling comment for clarity.
The original intent (warning about shutdown timeout) is still captured in
the code comment explaining potential issues.
2026-01-28 16:44:16 -08:00
Chris Lu
1e18c01a78 go fmt 2026-01-28 16:42:46 -08:00
Chris Lu
3e8d2a0a71 s3tables: Use policy_engine wildcard matcher for complete IAM compatibility
Replace the custom suffix-only wildcard implementation in matchesActionPattern
and matchesPrincipal with the policy_engine.MatchesWildcard function from
PR #8052. This enables full wildcard support including:

- Middle wildcards: s3tables:Get*Table matches GetTable
- Question mark wildcards: Get? matches any single character
- Combined patterns: s3tables:*Table* matches any action containing 'Table'

Benefits:
- Code reuse: eliminates duplicate wildcard logic
- Complete IAM compatibility: supports all AWS wildcard patterns
- Performance: uses efficient O(n) backtracking algorithm
- Consistency: same wildcard behavior across S3 API and S3 Tables

Add comprehensive unit tests covering exact matches, suffix wildcards,
middle wildcards, question marks, and combined patterns for both action
and principal matching.
2026-01-28 16:37:31 -08:00
Chris Lu
dbf6465b0e s3tables: Add log message when cluster shutdown times out
The timeout path (2 second wait for graceful shutdown) was silent. Add a
warning log message when it occurs to help diagnose flaky test issues and
indicate when the mini cluster didn't shut down cleanly.
2026-01-28 16:24:15 -08:00
Chris Lu
a27f6527ab s3tables: Extract resource owner and bucket extraction into helper method
Create extractResourceOwnerAndBucket() helper to consolidate the repeated pattern
of unmarshaling metadata and extracting bucket name from resource path. This
pattern was duplicated in handleTagResource, handleListTagsForResource, and
handleUntagResource. Update all three handlers to use the helper.

Also update remaining uses of getPrincipalFromRequest() (in handler_bucket_create,
handler_bucket_get_list_delete, handler_namespace) to use getAccountID() after
consolidating the two identical methods.
2026-01-28 16:24:07 -08:00
Chris Lu
0b41ade726 s3tables: Fetch bucket policy in handleListTagsForResource for permission evaluation
Update handleListTagsForResource to fetch and pass bucket policy to
CheckPermission, matching the behavior of handleTagResource/handleUntagResource.
This enables bucket-policy-based permission grants to be evaluated for
ListTagsForResource, not just ownership-based checks.
2026-01-28 16:23:12 -08:00
Chris Lu
41e799b4e0 s3tables: Consolidate getPrincipalFromRequest and getAccountID into single method
Both methods had identical implementations - they return the account ID from
request header or fall back to handler's default. Remove the duplicate
getPrincipalFromRequest and use getAccountID throughout, with updated comment
explaining its dual role as both caller identity and principal for permission
checks.
2026-01-28 16:23:01 -08:00
Chris Lu
ee3d779a5d s3tables: Separate permission checks for tagging and untagging
- Add CanTagResource() to check TagResource permission
- Add CanUntagResource() to check UntagResource permission
- Update CanManageTags() to check both operations (OR logic)

This prevents UntagResource from incorrectly checking 'ManageTags' permission
and ensures each operation validates the correct permission when per-operation
permissions are enforced.
2026-01-28 16:21:38 -08:00
Chris Lu
169ee629fa s3tables: Improve bucket name validation error message
Replace misleading character-only error message with generic 'invalid bucket
name'. The isValidBucketName() function checks multiple constraints beyond
character set (length, reserved prefixes/suffixes, start/end rules), so a
specific character message is inaccurate.
2026-01-28 16:21:15 -08:00
Chris Lu
fb8390c6a7 s3tables: Rename tableMetadataInternal.Schema to Metadata
The field name 'Schema' was confusing given it holds a *TableMetadata struct
and serializes as 'metadata' in JSON. Rename to 'Metadata' for clarity and
consistency with the JSON tag and intended meaning.
2026-01-28 16:21:06 -08:00
Chris Lu
191a858e72 s3tables: Fix parseTableFromARN() namespace and table name validation
- Remove dead URL unescape for namespace (regex [a-z0-9_]+ cannot contain
  percent-escapes)
- Add URL decoding and validation of extracted table name via
  validateTableName() to prevent callers from bypassing request validation
  done in other paths
2026-01-28 16:20:58 -08:00
Chris Lu
fb4fb8b082 s3tables: Validate bucket name in parseBucketNameFromARN()
Enforce the same bucket name validation rules (length, characters, reserved
prefixes/suffixes) when extracting from ARN. This prevents accepting ARNs
that the system would never create and ensures consistency with
CreateTableBucket validation.
2026-01-28 16:20:49 -08:00
Chris Lu
b1d7f3d6e8 s3tables: Add upper bound validation for MaxBuckets parameter
MaxBuckets is user-controlled and used in uint32(maxBuckets*2) for ListEntries.
Very large values can overflow uint32 or trigger overly expensive scans. Cap
MaxBuckets to 1000 and reject out-of-range values, consistent with MaxTables
handling and S3 MaxKeys validation elsewhere in the codebase.
2026-01-28 16:20:36 -08:00
Chris Lu
e0da63fd0a s3tables: Add upper bound validation for MaxTables parameter
MaxTables is user-controlled and influences gRPC ListEntries limits via
uint32(maxTables*2). Without an upper bound, very large values can overflow
uint32 or cause excessively large directory scans. Cap MaxTables to 1000 and
return InvalidRequest for out-of-range values, consistent with S3 MaxKeys
handling.
2026-01-28 16:20:32 -08:00
Chris Lu
2d556ac2a5 S3 Tables API now properly enforces resource policies
addressing the critical security gap where policies were created but never evaluated.
2026-01-28 16:15:34 -08:00
Chris Lu
e862888d2d s3tables: add request body size limiting
Add request body size limiting (10MB) to readRequestBody method:
- Define maxRequestBodySize constant to prevent unbounded reads
- Use io.LimitReader to enforce size limit
- Add explicit error handling for oversized requests
- Prevents potential DoS attacks via large request bodies
2026-01-28 14:54:45 -08:00
Chris Lu
b142689232 follow aws spec 2026-01-28 14:52:05 -08:00
Chris Lu
473e699368 s3tables: add table policy test coverage
Add comprehensive test coverage for table policy operations:
- Added PutTablePolicy, GetTablePolicy, DeleteTablePolicy methods to test client
- Implemented testTablePolicy lifecycle test validating Put/Get/Delete operations
- Verified error handling for missing policies
2026-01-28 14:44:28 -08:00
Chris Lu
0115e60919 s3tables: update bucket name validation message
Remove "underscores" from error message to accurately reflect that
bucket names only allow lowercase letters, numbers, and hyphens.
2026-01-28 14:41:15 -08:00
Chris Lu
a6c3e96f7b s3tables: fix double-write issue in handleListTables
Remove premature HTTP error writes from within WithFilerClient closure
to prevent duplicate status code responses. Error handling is now
consistently performed at the top level using isAuthError.
2026-01-28 14:41:14 -08:00
Chris Lu
dffe038efa go fmt 2026-01-28 14:34:07 -08:00
Chris Lu
4d4af0589b s3tables: standardize access denied errors using ErrAccessDenied constant 2026-01-28 14:33:01 -08:00
Chris Lu
d98e104dc5 s3tables: align ARN regex patterns with S3 standards and refactor to constants 2026-01-28 14:28:12 -08:00
Chris Lu
f5d71008d7 s3tables: refactor handleDeleteTableBucket to use strongly typed AuthError 2026-01-28 14:28:12 -08:00
Chris Lu
0d65daad4c s3tables: improve pagination robustness and error handling in table listing handlers 2026-01-28 14:04:09 -08:00
Chris Lu
612eae9ae8 s3tables: fix inconsistent permission check in handleCreateTableBucket 2026-01-28 14:04:08 -08:00
Chris Lu
a689c1e052 s3tables: align getPrincipalFromRequest with account ID for IAM compatibility 2026-01-28 14:04:08 -08:00
Chris Lu
d4ebafbacd s3tables: enforce strict resource ownership and implement result filtering for tables 2026-01-28 13:59:28 -08:00
Chris Lu
43aebc10da s3tables: enforce strict resource ownership and implement result filtering for namespaces 2026-01-28 13:59:24 -08:00
Chris Lu
9d54f4d160 s3tables: enforce strict resource ownership and implement result filtering for buckets 2026-01-28 13:59:21 -08:00
Chris Lu
c8cfbaa069 s3tables: implement strict AWS-aligned name validation for buckets, namespaces, and tables 2026-01-28 13:59:16 -08:00
Chris Lu
1697ec862f ownerAccountID 2026-01-28 13:54:49 -08:00
Chris Lu
1fdd9c3372 s3tables: refactor permission checks to use resource owner in policy and tagging handlers 2026-01-28 13:50:24 -08:00
Chris Lu
78a007d42b s3tables: refactor permission checks to use resource owner in table handlers 2026-01-28 13:50:19 -08:00
Chris Lu
ef0bae45e3 s3tables: refactor permission checks to use resource owner in namespace handlers 2026-01-28 13:50:16 -08:00
Chris Lu
32fade010a s3tables: refactor permission checks to use resource owner in bucket handlers 2026-01-28 13:50:13 -08:00
Chris Lu
2f4cee9538 s3tables: add isAuthError helper to handler.go 2026-01-28 13:50:10 -08:00
Chris Lu
090d473822 s3tables: allow hyphens in namespace and table names
Relaxed regex validation in utils.go to support hyphens in S3 Tables
namespaces and table names, improving consistency with S3 bucket naming
and allowing derived names from services like S3 Storage Lens.
2026-01-28 13:38:41 -08:00
Chris Lu
d6f6bf4ce7 s3tables: remove unused ExtractPrincipalFromContext function
Removed the unused ExtractPrincipalFromContext utility and its
accompanying iam/utils import to keep the new s3tables codebase clean.
2026-01-28 13:31:53 -08:00