Implement IAM propagation to S3 servers (#8130)
* Implement IAM propagation to S3 servers - Add PropagatingCredentialStore to propagate IAM changes to S3 servers via gRPC - Add Policy management RPCs to S3 proto and S3ApiServer - Update CredentialManager to use PropagatingCredentialStore when MasterClient is available - Wire FilerServer to enable propagation * Implement parallel IAM propagation and fix S3 cluster registration - Parallelized IAM change propagation with 10s timeout. - Refined context usage in PropagatingCredentialStore. - Added S3Type support to cluster node management. - Enabled S3 servers to register with gRPC address to the master. - Ensured IAM configuration reload after policy updates via gRPC. * Optimize IAM propagation with direct in-memory cache updates * Secure IAM propagation: Use metadata to skip persistence only on propagation * pb: refactor IAM and S3 services for unidirectional IAM propagation - Move SeaweedS3IamCache service from iam.proto to s3.proto. - Remove legacy IAM management RPCs and empty SeaweedS3 service from s3.proto. - Enforce that S3 servers only use the synchronization interface. * pb: regenerate Go code for IAM and S3 services Updated generated code following the proto refactoring of IAM synchronization services. * s3api: implement read-only mode for Embedded IAM API - Add readOnly flag to EmbeddedIamApi to reject write operations via HTTP. - Enable read-only mode by default in S3ApiServer. - Handle AccessDenied error in writeIamErrorResponse. - Embed SeaweedS3IamCacheServer in S3ApiServer. * credential: refactor PropagatingCredentialStore for unidirectional IAM flow - Update to use s3_pb.SeaweedS3IamCacheClient for propagation to S3 servers. - Propagate full Identity object via PutIdentity for consistency. - Remove redundant propagation of specific user/account/policy management RPCs. - Add timeout context for propagation calls. * s3api: implement SeaweedS3IamCacheServer for unidirectional sync - Update S3ApiServer to implement the cache synchronization gRPC interface. - Methods (PutIdentity, RemoveIdentity, etc.) now perform direct in-memory cache updates. - Register SeaweedS3IamCacheServer in command/s3.go. - Remove registration for the legacy and now empty SeaweedS3 service. * s3api: update tests for read-only IAM and propagation - Added TestEmbeddedIamReadOnly to verify rejection of write operations in read-only mode. - Update test setup to pass readOnly=false to NewEmbeddedIamApi in routing tests. - Updated EmbeddedIamApiForTest helper with read-only checks matching production behavior. * s3api: add back temporary debug logs for IAM updates Log IAM updates received via: - gRPC propagation (PutIdentity, PutPolicy, etc.) - Metadata configuration reloads (LoadS3ApiConfigurationFromCredentialManager) - Core identity management (UpsertIdentity, RemoveIdentity) * IAM: finalize propagation fix with reduced logging and clarified architecture * Allow configuring IAM read-only mode for S3 server integration tests * s3api: add defensive validation to UpsertIdentity * s3api: fix log message to reference correct IAM read-only flag * test/s3/iam: ensure WaitForS3Service checks for IAM write permissions * test: enable writable IAM in Makefile for integration tests * IAM: add GetPolicy/ListPolicies RPCs to s3.proto * S3: add GetBucketPolicy and ListBucketPolicies helpers * S3: support storing generic IAM policies in IdentityAccessManagement * S3: implement IAM policy RPCs using IdentityAccessManagement * IAM: fix stale user identity on rename propagation
This commit is contained in:
@@ -13,6 +13,9 @@ import (
|
||||
|
||||
"github.com/seaweedfs/seaweedfs/weed/pb/iam_pb"
|
||||
jsonpb "google.golang.org/protobuf/encoding/protojson"
|
||||
|
||||
_ "github.com/seaweedfs/seaweedfs/weed/credential/filer_etc"
|
||||
_ "github.com/seaweedfs/seaweedfs/weed/credential/memory"
|
||||
)
|
||||
|
||||
func TestIdentityListFileFormat(t *testing.T) {
|
||||
@@ -742,3 +745,48 @@ func TestSignatureVerificationDoesNotCheckPermissions(t *testing.T) {
|
||||
t.Log("Signature verification no longer checks for Write permission")
|
||||
t.Log("This allows list-only and read-only users to authenticate via AWS Signature V4")
|
||||
}
|
||||
|
||||
func TestStaticIdentityProtection(t *testing.T) {
|
||||
iam := NewIdentityAccessManagement(&S3ApiServerOption{})
|
||||
|
||||
// Add a static identity
|
||||
staticIdent := &Identity{
|
||||
Name: "static-user",
|
||||
IsStatic: true,
|
||||
}
|
||||
iam.m.Lock()
|
||||
if iam.nameToIdentity == nil {
|
||||
iam.nameToIdentity = make(map[string]*Identity)
|
||||
}
|
||||
iam.identities = append(iam.identities, staticIdent)
|
||||
iam.nameToIdentity[staticIdent.Name] = staticIdent
|
||||
iam.m.Unlock()
|
||||
|
||||
// Add a dynamic identity
|
||||
dynamicIdent := &Identity{
|
||||
Name: "dynamic-user",
|
||||
IsStatic: false,
|
||||
}
|
||||
iam.m.Lock()
|
||||
iam.identities = append(iam.identities, dynamicIdent)
|
||||
iam.nameToIdentity[dynamicIdent.Name] = dynamicIdent
|
||||
iam.m.Unlock()
|
||||
|
||||
// Try to remove static identity
|
||||
iam.RemoveIdentity("static-user")
|
||||
|
||||
// Verify static identity still exists
|
||||
iam.m.RLock()
|
||||
_, ok := iam.nameToIdentity["static-user"]
|
||||
iam.m.RUnlock()
|
||||
assert.True(t, ok, "Static identity should not be removed")
|
||||
|
||||
// Try to remove dynamic identity
|
||||
iam.RemoveIdentity("dynamic-user")
|
||||
|
||||
// Verify dynamic identity is removed
|
||||
iam.m.RLock()
|
||||
_, ok = iam.nameToIdentity["dynamic-user"]
|
||||
iam.m.RUnlock()
|
||||
assert.False(t, ok, "Dynamic identity should have been removed")
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user