s3: add s3:ExistingObjectTag condition support for bucket policies (#7677)

* s3: add s3:ExistingObjectTag condition support in policy engine

Add support for s3:ExistingObjectTag/<tag-key> condition keys in bucket
policies, allowing access control based on object tags.

Changes:
- Add ObjectEntry field to PolicyEvaluationArgs (entry.Extended metadata)
- Update EvaluateConditions to handle s3:ExistingObjectTag/<key> format
- Extract tag value from entry metadata using X-Amz-Tagging-<key> prefix

This enables policies like:
{
  "Condition": {
    "StringEquals": {
      "s3:ExistingObjectTag/status": ["public"]
    }
  }
}

Fixes: https://github.com/seaweedfs/seaweedfs/issues/7447

* s3: update EvaluatePolicy to accept object entry for tag conditions

Update BucketPolicyEngine.EvaluatePolicy to accept objectEntry parameter
(entry.Extended metadata) for evaluating tag-based policy conditions.

Changes:
- Add objectEntry parameter to EvaluatePolicy method
- Update callers in auth_credentials.go and s3api_bucket_handlers.go
- Pass nil for objectEntry in auth layer (entry fetched later in handlers)

For tag-based conditions to work, handlers should call EvaluatePolicy
with the object's entry.Extended after fetching the entry from filer.

* s3: add tests for s3:ExistingObjectTag policy conditions

Add comprehensive tests for object tag-based policy conditions:

- TestExistingObjectTagCondition: Basic tag matching scenarios
  - Matching/non-matching tag values
  - Missing tags, no tags, empty tags
  - Multiple tags with one matching

- TestExistingObjectTagConditionMultipleTags: Multiple tag conditions
  - Both tags match
  - Only one tag matches

- TestExistingObjectTagDenyPolicy: Deny policies with tag conditions
  - Default allow without tag
  - Deny when specific tag present

* s3: document s3:ExistingObjectTag support and feature status

Update policy engine documentation:

- Add s3:ExistingObjectTag/<tag-key> to supported condition keys
- Add 'Object Tag-Based Access Control' section with examples
- Add 'Feature Status' section with implemented and planned features

Planned features for future implementation:
- s3:RequestObjectTag/<key>
- s3:RequestObjectTagKeys
- s3:x-amz-server-side-encryption
- Cross-account access

* Implement tag-based policy re-check in handlers

- Add checkPolicyWithEntry helper to S3ApiServer for handlers to re-check
  policy after fetching object entry (for s3:ExistingObjectTag conditions)
- Add HasPolicyForBucket method to policy engine for efficient check
- Integrate policy re-check in GetObjectHandler after entry is fetched
- Integrate policy re-check in HeadObjectHandler after entry is fetched
- Update auth_credentials.go comments to explain two-phase evaluation
- Update documentation with supported operations for tag-based conditions

This implements 'Approach 1' where handlers re-check the policy with
the object entry after fetching it, allowing tag-based conditions to
be properly evaluated.

* Add integration tests for s3:ExistingObjectTag conditions

- Add TestCheckPolicyWithEntry: tests checkPolicyWithEntry helper with various
  tag scenarios (matching tags, non-matching tags, empty entry, nil entry)
- Add TestCheckPolicyWithEntryNoPolicyForBucket: tests early return when no policy
- Add TestCheckPolicyWithEntryNilPolicyEngine: tests nil engine handling
- Add TestCheckPolicyWithEntryDenyPolicy: tests deny policies with tag conditions
- Add TestHasPolicyForBucket: tests HasPolicyForBucket method

These tests cover the Phase 2 policy evaluation with object entry metadata,
ensuring tag-based conditions are properly evaluated.

* Address code review nitpicks

- Remove unused extractObjectTags placeholder function (engine.go)
- Add clarifying comment about s3:ExistingObjectTag/<key> evaluation
- Consolidate duplicate tag-based examples in README
- Factor out tagsToEntry helper to package level in tests

* Address code review feedback

- Fix unsafe type assertions in GetObjectHandler and HeadObjectHandler
  when getting identity from context (properly handle type assertion failure)
- Extract getConditionContextValue helper to eliminate duplicated logic
  between EvaluateConditions and EvaluateConditionsLegacy
- Ensure consistent handling of missing condition keys (always return
  empty slice)

* Fix GetObjectHandler to match HeadObjectHandler pattern

Add safety check for nil objectEntryForSSE before tag-based policy
evaluation, ensuring tag-based conditions are always evaluated rather
than silently skipped if entry is unexpectedly nil.

Addresses review comment from Copilot.

* Fix HeadObject action name in docs for consistency

Change 'HeadObject' to 's3:HeadObject' to match other action names.

* Extract recheckPolicyWithObjectEntry helper to reduce duplication

Move the repeated identity extraction and policy re-check logic from
GetObjectHandler and HeadObjectHandler into a shared helper method.

* Add validation for empty tag key in s3:ExistingObjectTag condition

Prevent potential issues with malformed policies containing
s3:ExistingObjectTag/ (empty tag key after slash).
This commit is contained in:
Chris Lu
2025-12-09 09:48:13 -08:00
committed by GitHub
parent d5f21fd8ba
commit d6d893c8c3
11 changed files with 761 additions and 104 deletions

View File

@@ -135,8 +135,70 @@ Standard AWS condition keys are supported:
- `aws:UserAgent` - Client user agent
- `s3:x-amz-acl` - Requested ACL
- `s3:VersionId` - Object version ID
- `s3:ExistingObjectTag/<tag-key>` - Value of an existing object tag (see example below)
- And many more...
### 5. Object Tag-Based Access Control
You can control access based on object tags using `s3:ExistingObjectTag/<tag-key>`:
```json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": "*",
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::my-bucket/*",
"Condition": {
"StringEquals": {
"s3:ExistingObjectTag/status": ["public"]
}
}
}
]
}
```
This allows anonymous access only to objects that have a tag `status=public`.
**Deny access to confidential objects:**
```json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": "*",
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::my-bucket/*"
},
{
"Effect": "Deny",
"Principal": "*",
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::my-bucket/*",
"Condition": {
"StringEquals": {
"s3:ExistingObjectTag/classification": ["confidential", "secret"]
}
}
}
]
}
```
**Supported Operations for Tag-Based Conditions:**
Tag-based conditions (`s3:ExistingObjectTag/<key>`) are evaluated for the following operations:
- `s3:GetObject` (GET object)
- `s3:GetObjectVersion` (GET object with versionId)
- `s3:HeadObject` (HEAD object)
Note: For these conditions to be evaluated, the object must exist and the policy engine re-checks access after fetching the object metadata.
## Policy Evaluation
### Evaluation Order (AWS-Compatible)
@@ -270,10 +332,39 @@ go test -v -run TestPolicyValidation
## Compatibility
- **Full backward compatibility** with existing `identities.json`
- **AWS S3 API compatibility** for bucket policies
- **Standard condition operators** and keys
- **Proper evaluation precedence** (Deny > Allow > Default Deny)
- **Performance optimized** with caching and compiled patterns
- Full backward compatibility with existing `identities.json`
- AWS S3 API compatibility for bucket policies
- Standard condition operators and keys
- Proper evaluation precedence (Deny > Allow > Default Deny)
- Performance optimized with caching and compiled patterns
The policy engine provides a seamless upgrade path from SeaweedFS's existing simple IAM system to full AWS S3-compatible policies, giving you the best of both worlds: simplicity for basic use cases and power for complex enterprise scenarios.
The policy engine provides a seamless upgrade path from SeaweedFS's existing simple IAM system to full AWS S3-compatible policies, giving you the best of both worlds: simplicity for basic use cases and power for complex enterprise scenarios.
## Feature Status
### Implemented
| Feature | Description |
|---------|-------------|
| Bucket Policies | Full AWS S3-compatible bucket policies |
| Condition Operators | StringEquals, IpAddress, Bool, DateGreaterThan, etc. |
| `aws:SourceIp` | IP-based access control with CIDR support |
| `aws:SecureTransport` | Require HTTPS |
| `aws:CurrentTime` | Time-based access control |
| `s3:ExistingObjectTag/<key>` | Tag-based access control for existing objects |
| Wildcard Patterns | Support for `*` and `?` in actions and resources |
| Principal Matching | `*`, account IDs, and user ARNs |
### Planned
| Feature | GitHub Issue |
|---------|--------------|
| `s3:RequestObjectTag/<key>` | For tag conditions on PUT requests |
| `s3:RequestObjectTagKeys` | Check which tag keys are in request |
| `s3:x-amz-content-sha256` | Content hash condition |
| `s3:x-amz-server-side-encryption` | SSE condition |
| `s3:x-amz-storage-class` | Storage class condition |
| Cross-account access | Access across different accounts |
| VPC Endpoint policies | Network-level policies |
For feature requests or to track progress, see the [GitHub Issues](https://github.com/seaweedfs/seaweedfs/issues).

View File

@@ -10,6 +10,7 @@ import (
"time"
"github.com/seaweedfs/seaweedfs/weed/glog"
"github.com/seaweedfs/seaweedfs/weed/s3api/s3_constants"
)
// LRUNode represents a node in the doubly-linked list for efficient LRU operations
@@ -705,8 +706,36 @@ func GetConditionEvaluator(operator string) (ConditionEvaluator, error) {
}
}
// ExistingObjectTagPrefix is the prefix for S3 policy condition keys
const ExistingObjectTagPrefix = "s3:ExistingObjectTag/"
// getConditionContextValue resolves the value(s) for a condition key.
// For s3:ExistingObjectTag/<key> conditions, it looks up the tag in objectEntry.
// For other condition keys, it looks up the value in contextValues.
func getConditionContextValue(key string, contextValues map[string][]string, objectEntry map[string][]byte) []string {
if strings.HasPrefix(key, ExistingObjectTagPrefix) {
tagKey := key[len(ExistingObjectTagPrefix):]
if tagKey == "" {
return []string{} // Invalid: empty tag key
}
metadataKey := s3_constants.AmzObjectTaggingPrefix + tagKey
if objectEntry != nil {
if tagValue, exists := objectEntry[metadataKey]; exists {
return []string{string(tagValue)}
}
}
return []string{}
}
if vals, exists := contextValues[key]; exists {
return vals
}
return []string{}
}
// EvaluateConditions evaluates all conditions in a policy statement
func EvaluateConditions(conditions PolicyConditions, contextValues map[string][]string) bool {
// objectEntry is the object's metadata from entry.Extended (can be nil)
func EvaluateConditions(conditions PolicyConditions, contextValues map[string][]string, objectEntry map[string][]byte) bool {
if len(conditions) == 0 {
return true // No conditions means always true
}
@@ -719,11 +748,7 @@ func EvaluateConditions(conditions PolicyConditions, contextValues map[string][]
}
for key, value := range conditionMap {
contextVals, exists := contextValues[key]
if !exists {
contextVals = []string{}
}
contextVals := getConditionContextValue(key, contextValues, objectEntry)
if !conditionEvaluator.Evaluate(value.Strings(), contextVals) {
return false // If any condition fails, the whole condition block fails
}
@@ -734,7 +759,8 @@ func EvaluateConditions(conditions PolicyConditions, contextValues map[string][]
}
// EvaluateConditionsLegacy evaluates conditions using the old interface{} format for backward compatibility
func EvaluateConditionsLegacy(conditions map[string]interface{}, contextValues map[string][]string) bool {
// objectEntry is the object's metadata from entry.Extended (can be nil)
func EvaluateConditionsLegacy(conditions map[string]interface{}, contextValues map[string][]string, objectEntry map[string][]byte) bool {
if len(conditions) == 0 {
return true // No conditions means always true
}
@@ -753,11 +779,7 @@ func EvaluateConditionsLegacy(conditions map[string]interface{}, contextValues m
}
for key, value := range conditionMapTyped {
contextVals, exists := contextValues[key]
if !exists {
contextVals = []string{}
}
contextVals := getConditionContextValue(key, contextValues, objectEntry)
if !conditionEvaluator.Evaluate(value, contextVals) {
return false // If any condition fails, the whole condition block fails
}

View File

@@ -91,6 +91,14 @@ func (engine *PolicyEngine) DeleteBucketPolicy(bucketName string) error {
return nil
}
// HasPolicyForBucket checks if a bucket has a policy configured
func (engine *PolicyEngine) HasPolicyForBucket(bucketName string) bool {
engine.mutex.RLock()
defer engine.mutex.RUnlock()
_, exists := engine.contexts[bucketName]
return exists
}
// EvaluatePolicy evaluates a policy for the given arguments
func (engine *PolicyEngine) EvaluatePolicy(bucketName string, args *PolicyEvaluationArgs) PolicyEvaluationResult {
engine.mutex.RLock()
@@ -154,7 +162,7 @@ func (engine *PolicyEngine) evaluateStatement(stmt *CompiledStatement, args *Pol
// Check conditions
if len(stmt.Statement.Condition) > 0 {
if !EvaluateConditions(stmt.Statement.Condition, args.Conditions) {
if !EvaluateConditions(stmt.Statement.Condition, args.Conditions, args.ObjectEntry) {
return false
}
}
@@ -201,10 +209,8 @@ func ExtractConditionValuesFromRequest(r *http.Request) map[string][]string {
values["aws:Referer"] = []string{referer}
}
// S3 object-level conditions
if r.Method == "GET" || r.Method == "HEAD" {
values["s3:ExistingObjectTag"] = extractObjectTags(r)
}
// Note: s3:ExistingObjectTag/<key> conditions are evaluated using objectEntry
// passed to EvaluatePolicy, not extracted from the request.
// S3 bucket-level conditions
if delimiter := r.URL.Query().Get("delimiter"); delimiter != "" {
@@ -243,13 +249,6 @@ func ExtractConditionValuesFromRequest(r *http.Request) map[string][]string {
return values
}
// extractObjectTags extracts object tags from request (placeholder implementation)
func extractObjectTags(r *http.Request) []string {
// This would need to be implemented based on how object tags are stored
// For now, return empty slice
return []string{}
}
// BuildResourceArn builds an ARN for the given bucket and object
func BuildResourceArn(bucketName, objectName string) string {
if objectName == "" {
@@ -352,15 +351,6 @@ func GetObjectNameFromArn(arn string) string {
return ""
}
// HasPolicyForBucket checks if a bucket has a policy
func (engine *PolicyEngine) HasPolicyForBucket(bucketName string) bool {
engine.mutex.RLock()
defer engine.mutex.RUnlock()
_, exists := engine.contexts[bucketName]
return exists
}
// GetPolicyStatements returns all policy statements for a bucket
func (engine *PolicyEngine) GetPolicyStatements(bucketName string) []PolicyStatement {
engine.mutex.RLock()

View File

@@ -5,9 +5,23 @@ import (
"net/url"
"testing"
"github.com/seaweedfs/seaweedfs/weed/s3api/s3_constants"
"github.com/seaweedfs/seaweedfs/weed/s3api/s3err"
)
// tagsToEntry converts a map of tag key-value pairs to the entry.Extended format
// used for s3:ExistingObjectTag/<key> condition evaluation
func tagsToEntry(tags map[string]string) map[string][]byte {
if tags == nil {
return nil
}
entry := make(map[string][]byte)
for k, v := range tags {
entry[s3_constants.AmzObjectTaggingPrefix+k] = []byte(v)
}
return entry
}
func TestPolicyEngine(t *testing.T) {
engine := NewPolicyEngine()
@@ -714,3 +728,233 @@ type MockLegacyIAM struct{}
func (m *MockLegacyIAM) authRequest(r *http.Request, action Action) (Identity, s3err.ErrorCode) {
return nil, s3err.ErrNone
}
// TestExistingObjectTagCondition tests s3:ExistingObjectTag/<tag-key> condition support
func TestExistingObjectTagCondition(t *testing.T) {
engine := NewPolicyEngine()
// Policy that allows GetObject only for objects with specific tag
policyJSON := `{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": "*",
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::test-bucket/*",
"Condition": {
"StringEquals": {
"s3:ExistingObjectTag/status": ["public"]
}
}
}
]
}`
err := engine.SetBucketPolicy("test-bucket", policyJSON)
if err != nil {
t.Fatalf("Failed to set bucket policy: %v", err)
}
tests := []struct {
name string
objectTags map[string]string
expected PolicyEvaluationResult
}{
{
name: "Matching tag value - should allow",
objectTags: map[string]string{"status": "public"},
expected: PolicyResultAllow,
},
{
name: "Non-matching tag value - should be indeterminate",
objectTags: map[string]string{"status": "private"},
expected: PolicyResultIndeterminate,
},
{
name: "Missing tag - should be indeterminate",
objectTags: map[string]string{"other": "value"},
expected: PolicyResultIndeterminate,
},
{
name: "No tags - should be indeterminate",
objectTags: nil,
expected: PolicyResultIndeterminate,
},
{
name: "Empty tags - should be indeterminate",
objectTags: map[string]string{},
expected: PolicyResultIndeterminate,
},
{
name: "Multiple tags with matching one - should allow",
objectTags: map[string]string{"status": "public", "owner": "admin"},
expected: PolicyResultAllow,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
args := &PolicyEvaluationArgs{
Action: "s3:GetObject",
Resource: "arn:aws:s3:::test-bucket/test-object",
Principal: "*",
ObjectEntry: tagsToEntry(tt.objectTags),
}
result := engine.EvaluatePolicy("test-bucket", args)
if result != tt.expected {
t.Errorf("Expected %v, got %v", tt.expected, result)
}
})
}
}
// TestExistingObjectTagConditionMultipleTags tests policies with multiple tag conditions
func TestExistingObjectTagConditionMultipleTags(t *testing.T) {
engine := NewPolicyEngine()
// Policy that requires multiple tag conditions
policyJSON := `{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": "*",
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::test-bucket/*",
"Condition": {
"StringEquals": {
"s3:ExistingObjectTag/status": ["public"],
"s3:ExistingObjectTag/tier": ["free", "premium"]
}
}
}
]
}`
err := engine.SetBucketPolicy("test-bucket", policyJSON)
if err != nil {
t.Fatalf("Failed to set bucket policy: %v", err)
}
tests := []struct {
name string
objectTags map[string]string
expected PolicyEvaluationResult
}{
{
name: "Both tags match - should allow",
objectTags: map[string]string{"status": "public", "tier": "free"},
expected: PolicyResultAllow,
},
{
name: "Both tags match (premium tier) - should allow",
objectTags: map[string]string{"status": "public", "tier": "premium"},
expected: PolicyResultAllow,
},
{
name: "Only status matches - should be indeterminate",
objectTags: map[string]string{"status": "public"},
expected: PolicyResultIndeterminate,
},
{
name: "Only tier matches - should be indeterminate",
objectTags: map[string]string{"tier": "free"},
expected: PolicyResultIndeterminate,
},
{
name: "Neither tag matches - should be indeterminate",
objectTags: map[string]string{"status": "private", "tier": "basic"},
expected: PolicyResultIndeterminate,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
args := &PolicyEvaluationArgs{
Action: "s3:GetObject",
Resource: "arn:aws:s3:::test-bucket/test-object",
Principal: "*",
ObjectEntry: tagsToEntry(tt.objectTags),
}
result := engine.EvaluatePolicy("test-bucket", args)
if result != tt.expected {
t.Errorf("Expected %v, got %v", tt.expected, result)
}
})
}
}
// TestExistingObjectTagDenyPolicy tests deny policies with tag conditions
func TestExistingObjectTagDenyPolicy(t *testing.T) {
engine := NewPolicyEngine()
// Policy that denies access to objects with confidential tag
policyJSON := `{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": "*",
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::test-bucket/*"
},
{
"Effect": "Deny",
"Principal": "*",
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::test-bucket/*",
"Condition": {
"StringEquals": {
"s3:ExistingObjectTag/classification": ["confidential"]
}
}
}
]
}`
err := engine.SetBucketPolicy("test-bucket", policyJSON)
if err != nil {
t.Fatalf("Failed to set bucket policy: %v", err)
}
tests := []struct {
name string
objectTags map[string]string
expected PolicyEvaluationResult
}{
{
name: "No tags - allow by default statement",
objectTags: nil,
expected: PolicyResultAllow,
},
{
name: "Non-confidential tag - allow",
objectTags: map[string]string{"classification": "public"},
expected: PolicyResultAllow,
},
{
name: "Confidential tag - deny",
objectTags: map[string]string{"classification": "confidential"},
expected: PolicyResultDeny,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
args := &PolicyEvaluationArgs{
Action: "s3:GetObject",
Resource: "arn:aws:s3:::test-bucket/test-object",
Principal: "*",
ObjectEntry: tagsToEntry(tt.objectTags),
}
result := engine.EvaluatePolicy("test-bucket", args)
if result != tt.expected {
t.Errorf("Expected %v, got %v", tt.expected, result)
}
})
}
}

View File

@@ -106,6 +106,11 @@ type PolicyEvaluationArgs struct {
Resource string
Principal string
Conditions map[string][]string
// ObjectEntry is the object's metadata from entry.Extended.
// Used for evaluating conditions like s3:ExistingObjectTag/<tag-key>.
// Tags are stored with s3_constants.AmzObjectTaggingPrefix (X-Amz-Tagging-) prefix.
// Can be nil for bucket-level operations or when object doesn't exist.
ObjectEntry map[string][]byte
}
// PolicyCache for caching compiled policies