Files
seaweedFS/weed/s3api/s3api_server_grpc.go
Chris Lu 551a31e156 Implement IAM propagation to S3 servers (#8130)
* Implement IAM propagation to S3 servers

- Add PropagatingCredentialStore to propagate IAM changes to S3 servers via gRPC
- Add Policy management RPCs to S3 proto and S3ApiServer
- Update CredentialManager to use PropagatingCredentialStore when MasterClient is available
- Wire FilerServer to enable propagation

* Implement parallel IAM propagation and fix S3 cluster registration

- Parallelized IAM change propagation with 10s timeout.
- Refined context usage in PropagatingCredentialStore.
- Added S3Type support to cluster node management.
- Enabled S3 servers to register with gRPC address to the master.
- Ensured IAM configuration reload after policy updates via gRPC.

* Optimize IAM propagation with direct in-memory cache updates

* Secure IAM propagation: Use metadata to skip persistence only on propagation

* pb: refactor IAM and S3 services for unidirectional IAM propagation

- Move SeaweedS3IamCache service from iam.proto to s3.proto.
- Remove legacy IAM management RPCs and empty SeaweedS3 service from s3.proto.
- Enforce that S3 servers only use the synchronization interface.

* pb: regenerate Go code for IAM and S3 services

Updated generated code following the proto refactoring of IAM synchronization services.

* s3api: implement read-only mode for Embedded IAM API

- Add readOnly flag to EmbeddedIamApi to reject write operations via HTTP.
- Enable read-only mode by default in S3ApiServer.
- Handle AccessDenied error in writeIamErrorResponse.
- Embed SeaweedS3IamCacheServer in S3ApiServer.

* credential: refactor PropagatingCredentialStore for unidirectional IAM flow

- Update to use s3_pb.SeaweedS3IamCacheClient for propagation to S3 servers.
- Propagate full Identity object via PutIdentity for consistency.
- Remove redundant propagation of specific user/account/policy management RPCs.
- Add timeout context for propagation calls.

* s3api: implement SeaweedS3IamCacheServer for unidirectional sync

- Update S3ApiServer to implement the cache synchronization gRPC interface.
- Methods (PutIdentity, RemoveIdentity, etc.) now perform direct in-memory cache updates.
- Register SeaweedS3IamCacheServer in command/s3.go.
- Remove registration for the legacy and now empty SeaweedS3 service.

* s3api: update tests for read-only IAM and propagation

- Added TestEmbeddedIamReadOnly to verify rejection of write operations in read-only mode.
- Update test setup to pass readOnly=false to NewEmbeddedIamApi in routing tests.
- Updated EmbeddedIamApiForTest helper with read-only checks matching production behavior.

* s3api: add back temporary debug logs for IAM updates

Log IAM updates received via:
- gRPC propagation (PutIdentity, PutPolicy, etc.)
- Metadata configuration reloads (LoadS3ApiConfigurationFromCredentialManager)
- Core identity management (UpsertIdentity, RemoveIdentity)

* IAM: finalize propagation fix with reduced logging and clarified architecture

* Allow configuring IAM read-only mode for S3 server integration tests

* s3api: add defensive validation to UpsertIdentity

* s3api: fix log message to reference correct IAM read-only flag

* test/s3/iam: ensure WaitForS3Service checks for IAM write permissions

* test: enable writable IAM in Makefile for integration tests

* IAM: add GetPolicy/ListPolicies RPCs to s3.proto

* S3: add GetBucketPolicy and ListBucketPolicies helpers

* S3: support storing generic IAM policies in IdentityAccessManagement

* S3: implement IAM policy RPCs using IdentityAccessManagement

* IAM: fix stale user identity on rename propagation
2026-01-26 22:59:43 -08:00

99 lines
3.1 KiB
Go

package s3api
import (
"context"
"fmt"
"github.com/seaweedfs/seaweedfs/weed/glog"
"github.com/seaweedfs/seaweedfs/weed/pb/iam_pb"
)
// SeaweedS3IamCacheServer Implementation
// This interface is dedicated to UNIDIRECTIONAL updates from Filer to S3 Server.
// S3 Server acts purely as a cache.
func (s3a *S3ApiServer) PutIdentity(ctx context.Context, req *iam_pb.PutIdentityRequest) (*iam_pb.PutIdentityResponse, error) {
if req.Identity == nil {
return nil, fmt.Errorf("identity is required")
}
// Direct in-memory cache update
glog.V(1).Infof("IAM: received identity update for %s", req.Identity.Name)
if err := s3a.iam.UpsertIdentity(req.Identity); err != nil {
glog.Errorf("failed to update identity cache for %s: %v", req.Identity.Name, err)
return nil, err
}
return &iam_pb.PutIdentityResponse{}, nil
}
func (s3a *S3ApiServer) RemoveIdentity(ctx context.Context, req *iam_pb.RemoveIdentityRequest) (*iam_pb.RemoveIdentityResponse, error) {
if req.Username == "" {
return nil, fmt.Errorf("username is required")
}
// Direct in-memory cache update
glog.V(1).Infof("IAM: received identity removal for %s", req.Username)
s3a.iam.RemoveIdentity(req.Username)
return &iam_pb.RemoveIdentityResponse{}, nil
}
func (s3a *S3ApiServer) PutPolicy(ctx context.Context, req *iam_pb.PutPolicyRequest) (*iam_pb.PutPolicyResponse, error) {
if req.Name == "" {
return nil, fmt.Errorf("policy name is required")
}
// Update IAM policy cache
glog.V(1).Infof("IAM: received policy update for %s", req.Name)
if s3a.iam != nil {
if err := s3a.iam.PutPolicy(req.Name, req.Content); err != nil {
glog.Errorf("failed to update policy cache for %s: %v", req.Name, err)
return nil, err
}
}
return &iam_pb.PutPolicyResponse{}, nil
}
func (s3a *S3ApiServer) DeletePolicy(ctx context.Context, req *iam_pb.DeletePolicyRequest) (*iam_pb.DeletePolicyResponse, error) {
if req.Name == "" {
return nil, fmt.Errorf("policy name is required")
}
// Delete from IAM policy cache
glog.V(1).Infof("IAM: received policy removal for %s", req.Name)
if s3a.iam != nil {
if err := s3a.iam.DeletePolicy(req.Name); err != nil {
glog.Errorf("failed to delete policy cache for %s: %v", req.Name, err)
return nil, err
}
}
return &iam_pb.DeletePolicyResponse{}, nil
}
func (s3a *S3ApiServer) GetPolicy(ctx context.Context, req *iam_pb.GetPolicyRequest) (*iam_pb.GetPolicyResponse, error) {
if req.Name == "" {
return nil, fmt.Errorf("policy name is required")
}
if s3a.iam == nil {
return &iam_pb.GetPolicyResponse{}, nil
}
policy, err := s3a.iam.GetPolicy(req.Name)
if err != nil {
return &iam_pb.GetPolicyResponse{}, nil // Not found is fine for cache
}
return &iam_pb.GetPolicyResponse{
Name: policy.Name,
Content: policy.Content,
}, nil
}
func (s3a *S3ApiServer) ListPolicies(ctx context.Context, req *iam_pb.ListPoliciesRequest) (*iam_pb.ListPoliciesResponse, error) {
resp := &iam_pb.ListPoliciesResponse{}
if s3a.iam == nil {
return resp, nil
}
policies := s3a.iam.ListPolicies()
for _, policy := range policies {
resp.Policies = append(resp.Policies, policy)
}
return resp, nil
}