Commit Graph

7 Commits

Author SHA1 Message Date
Chris Lu
92800c31a2 adjust logs and errors 2026-01-27 07:45:24 -08:00
Chris Lu
551a31e156 Implement IAM propagation to S3 servers (#8130)
* Implement IAM propagation to S3 servers

- Add PropagatingCredentialStore to propagate IAM changes to S3 servers via gRPC
- Add Policy management RPCs to S3 proto and S3ApiServer
- Update CredentialManager to use PropagatingCredentialStore when MasterClient is available
- Wire FilerServer to enable propagation

* Implement parallel IAM propagation and fix S3 cluster registration

- Parallelized IAM change propagation with 10s timeout.
- Refined context usage in PropagatingCredentialStore.
- Added S3Type support to cluster node management.
- Enabled S3 servers to register with gRPC address to the master.
- Ensured IAM configuration reload after policy updates via gRPC.

* Optimize IAM propagation with direct in-memory cache updates

* Secure IAM propagation: Use metadata to skip persistence only on propagation

* pb: refactor IAM and S3 services for unidirectional IAM propagation

- Move SeaweedS3IamCache service from iam.proto to s3.proto.
- Remove legacy IAM management RPCs and empty SeaweedS3 service from s3.proto.
- Enforce that S3 servers only use the synchronization interface.

* pb: regenerate Go code for IAM and S3 services

Updated generated code following the proto refactoring of IAM synchronization services.

* s3api: implement read-only mode for Embedded IAM API

- Add readOnly flag to EmbeddedIamApi to reject write operations via HTTP.
- Enable read-only mode by default in S3ApiServer.
- Handle AccessDenied error in writeIamErrorResponse.
- Embed SeaweedS3IamCacheServer in S3ApiServer.

* credential: refactor PropagatingCredentialStore for unidirectional IAM flow

- Update to use s3_pb.SeaweedS3IamCacheClient for propagation to S3 servers.
- Propagate full Identity object via PutIdentity for consistency.
- Remove redundant propagation of specific user/account/policy management RPCs.
- Add timeout context for propagation calls.

* s3api: implement SeaweedS3IamCacheServer for unidirectional sync

- Update S3ApiServer to implement the cache synchronization gRPC interface.
- Methods (PutIdentity, RemoveIdentity, etc.) now perform direct in-memory cache updates.
- Register SeaweedS3IamCacheServer in command/s3.go.
- Remove registration for the legacy and now empty SeaweedS3 service.

* s3api: update tests for read-only IAM and propagation

- Added TestEmbeddedIamReadOnly to verify rejection of write operations in read-only mode.
- Update test setup to pass readOnly=false to NewEmbeddedIamApi in routing tests.
- Updated EmbeddedIamApiForTest helper with read-only checks matching production behavior.

* s3api: add back temporary debug logs for IAM updates

Log IAM updates received via:
- gRPC propagation (PutIdentity, PutPolicy, etc.)
- Metadata configuration reloads (LoadS3ApiConfigurationFromCredentialManager)
- Core identity management (UpsertIdentity, RemoveIdentity)

* IAM: finalize propagation fix with reduced logging and clarified architecture

* Allow configuring IAM read-only mode for S3 server integration tests

* s3api: add defensive validation to UpsertIdentity

* s3api: fix log message to reference correct IAM read-only flag

* test/s3/iam: ensure WaitForS3Service checks for IAM write permissions

* test: enable writable IAM in Makefile for integration tests

* IAM: add GetPolicy/ListPolicies RPCs to s3.proto

* S3: add GetBucketPolicy and ListBucketPolicies helpers

* S3: support storing generic IAM policies in IdentityAccessManagement

* S3: implement IAM policy RPCs using IdentityAccessManagement

* IAM: fix stale user identity on rename propagation
2026-01-26 22:59:43 -08:00
Chris Lu
43229b05ce Explicit IAM gRPC APIs for S3 Server (#8126)
* Update IAM and S3 protobuf definitions for explicit IAM gRPC APIs

* Refactor s3api: Extract generic ExecuteAction method for IAM operations

* Implement explicit IAM gRPC APIs in S3 server

* iam: remove deprecated GetConfiguration and PutConfiguration RPCs

* iamapi: refactor handlers to use CredentialManager directly

* s3api: refactor embedded IAM to use CredentialManager directly

* server: remove deprecated configuration gRPC handlers

* credential/grpc: refactor configuration calls to return error

* shell: update s3.configure to list users instead of full config

* s3api: fix CreateServiceAccount gRPC handler to map required fields

* s3api: fix UpdateServiceAccount gRPC handler to map fields and safe status

* s3api: enforce UserName in embedded IAM ListAccessKeys

* test: fix test_config.json structure to match proto definition

* Revert "credential/grpc: refactor configuration calls to return error"

This reverts commit cde707dd8b88c7d1bd730271518542eceb5ed069.

* Revert "server: remove deprecated configuration gRPC handlers"

This reverts commit 7307e205a083c8315cf84ddc2614b3e50eda2e33.

* Revert "s3api: enforce UserName in embedded IAM ListAccessKeys"

This reverts commit adf727ba52b4f3ffb911f0d0df85db858412ff83.

* Revert "s3api: fix UpdateServiceAccount gRPC handler to map fields and safe status"

This reverts commit 6a4be3314d43b6c8fda8d5e0558e83e87a19df3f.

* Revert "s3api: fix CreateServiceAccount gRPC handler to map required fields"

This reverts commit 9bb4425f07fbad38fb68d33e5c0aa573d8912a37.

* Revert "shell: update s3.configure to list users instead of full config"

This reverts commit f3304ead537b3e6be03d46df4cb55983ab931726.

* Revert "s3api: refactor embedded IAM to use CredentialManager directly"

This reverts commit 9012f27af82d11f0e824877712a5ae2505a65f86.

* Revert "iamapi: refactor handlers to use CredentialManager directly"

This reverts commit 3a148212236576b0a3aa4d991c2abb014fb46091.

* Revert "iam: remove deprecated GetConfiguration and PutConfiguration RPCs"

This reverts commit e16e08aa0099699338d3155bc7428e1051ce0a6a.

* s3api: address IAM code review comments (error handling, logging, gRPC response mapping)

* s3api: add robustness to startup by retrying KEK and IAM config loading from Filer

* s3api: address IAM gRPC code review comments (safety, validation, status logic)

* fix return
2026-01-26 13:38:15 -08:00
promalert
9012069bd7 chore: execute goimports to format the code (#7983)
* chore: execute goimports to format the code

Signed-off-by: promalert <promalert@outlook.com>

* goimports -w .

---------

Signed-off-by: promalert <promalert@outlook.com>
Co-authored-by: Chris Lu <chris.lu@gmail.com>
2026-01-07 13:06:08 -08:00
chrislu
26dbc6c905 move to https://github.com/seaweedfs/seaweedfs 2022-07-29 00:17:28 -07:00
chrislu
866fed1eab avoid grpc name conflict
fix https://github.com/chrislusf/seaweedfs/issues/3055
2022-05-15 21:31:29 -07:00
chrislu
29198720f2 s3: add grpc server to accept configuration changes 2022-05-15 00:43:37 -07:00