Files
seaweedFS/weed/pb/iam.proto
Chris Lu 551a31e156 Implement IAM propagation to S3 servers (#8130)
* Implement IAM propagation to S3 servers

- Add PropagatingCredentialStore to propagate IAM changes to S3 servers via gRPC
- Add Policy management RPCs to S3 proto and S3ApiServer
- Update CredentialManager to use PropagatingCredentialStore when MasterClient is available
- Wire FilerServer to enable propagation

* Implement parallel IAM propagation and fix S3 cluster registration

- Parallelized IAM change propagation with 10s timeout.
- Refined context usage in PropagatingCredentialStore.
- Added S3Type support to cluster node management.
- Enabled S3 servers to register with gRPC address to the master.
- Ensured IAM configuration reload after policy updates via gRPC.

* Optimize IAM propagation with direct in-memory cache updates

* Secure IAM propagation: Use metadata to skip persistence only on propagation

* pb: refactor IAM and S3 services for unidirectional IAM propagation

- Move SeaweedS3IamCache service from iam.proto to s3.proto.
- Remove legacy IAM management RPCs and empty SeaweedS3 service from s3.proto.
- Enforce that S3 servers only use the synchronization interface.

* pb: regenerate Go code for IAM and S3 services

Updated generated code following the proto refactoring of IAM synchronization services.

* s3api: implement read-only mode for Embedded IAM API

- Add readOnly flag to EmbeddedIamApi to reject write operations via HTTP.
- Enable read-only mode by default in S3ApiServer.
- Handle AccessDenied error in writeIamErrorResponse.
- Embed SeaweedS3IamCacheServer in S3ApiServer.

* credential: refactor PropagatingCredentialStore for unidirectional IAM flow

- Update to use s3_pb.SeaweedS3IamCacheClient for propagation to S3 servers.
- Propagate full Identity object via PutIdentity for consistency.
- Remove redundant propagation of specific user/account/policy management RPCs.
- Add timeout context for propagation calls.

* s3api: implement SeaweedS3IamCacheServer for unidirectional sync

- Update S3ApiServer to implement the cache synchronization gRPC interface.
- Methods (PutIdentity, RemoveIdentity, etc.) now perform direct in-memory cache updates.
- Register SeaweedS3IamCacheServer in command/s3.go.
- Remove registration for the legacy and now empty SeaweedS3 service.

* s3api: update tests for read-only IAM and propagation

- Added TestEmbeddedIamReadOnly to verify rejection of write operations in read-only mode.
- Update test setup to pass readOnly=false to NewEmbeddedIamApi in routing tests.
- Updated EmbeddedIamApiForTest helper with read-only checks matching production behavior.

* s3api: add back temporary debug logs for IAM updates

Log IAM updates received via:
- gRPC propagation (PutIdentity, PutPolicy, etc.)
- Metadata configuration reloads (LoadS3ApiConfigurationFromCredentialManager)
- Core identity management (UpsertIdentity, RemoveIdentity)

* IAM: finalize propagation fix with reduced logging and clarified architecture

* Allow configuring IAM read-only mode for S3 server integration tests

* s3api: add defensive validation to UpsertIdentity

* s3api: fix log message to reference correct IAM read-only flag

* test/s3/iam: ensure WaitForS3Service checks for IAM write permissions

* test: enable writable IAM in Makefile for integration tests

* IAM: add GetPolicy/ListPolicies RPCs to s3.proto

* S3: add GetBucketPolicy and ListBucketPolicies helpers

* S3: support storing generic IAM policies in IdentityAccessManagement

* S3: implement IAM policy RPCs using IdentityAccessManagement

* IAM: fix stale user identity on rename propagation
2026-01-26 22:59:43 -08:00

312 lines
7.4 KiB
Protocol Buffer

syntax = "proto3";
package iam_pb;
option go_package = "github.com/seaweedfs/seaweedfs/weed/pb/iam_pb";
option java_package = "seaweedfs.client";
option java_outer_classname = "IamProto";
//////////////////////////////////////////////////
service SeaweedIdentityAccessManagement {
// Configuration Management
rpc GetConfiguration (GetConfigurationRequest) returns (GetConfigurationResponse);
rpc PutConfiguration (PutConfigurationRequest) returns (PutConfigurationResponse);
// User Management
rpc CreateUser (CreateUserRequest) returns (CreateUserResponse);
rpc GetUser (GetUserRequest) returns (GetUserResponse);
rpc UpdateUser (UpdateUserRequest) returns (UpdateUserResponse);
rpc DeleteUser (DeleteUserRequest) returns (DeleteUserResponse);
rpc ListUsers (ListUsersRequest) returns (ListUsersResponse);
// Access Key Management
rpc CreateAccessKey (CreateAccessKeyRequest) returns (CreateAccessKeyResponse);
rpc DeleteAccessKey (DeleteAccessKeyRequest) returns (DeleteAccessKeyResponse);
rpc GetUserByAccessKey (GetUserByAccessKeyRequest) returns (GetUserByAccessKeyResponse);
// Policy Management
rpc PutPolicy (PutPolicyRequest) returns (PutPolicyResponse);
rpc GetPolicy (GetPolicyRequest) returns (GetPolicyResponse);
rpc ListPolicies (ListPoliciesRequest) returns (ListPoliciesResponse);
rpc DeletePolicy (DeletePolicyRequest) returns (DeletePolicyResponse);
// Service Account Management
rpc CreateServiceAccount (CreateServiceAccountRequest) returns (CreateServiceAccountResponse);
rpc UpdateServiceAccount (UpdateServiceAccountRequest) returns (UpdateServiceAccountResponse);
rpc DeleteServiceAccount (DeleteServiceAccountRequest) returns (DeleteServiceAccountResponse);
rpc GetServiceAccount (GetServiceAccountRequest) returns (GetServiceAccountResponse);
rpc ListServiceAccounts (ListServiceAccountsRequest) returns (ListServiceAccountsResponse);
rpc GetServiceAccountByAccessKey (GetServiceAccountByAccessKeyRequest) returns (GetServiceAccountByAccessKeyResponse);
}
//////////////////////////////////////////////////
// Configuration Management Messages
message GetConfigurationRequest {
}
message GetConfigurationResponse {
S3ApiConfiguration configuration = 1;
}
message PutConfigurationRequest {
S3ApiConfiguration configuration = 1;
}
message PutConfigurationResponse {
}
//////////////////////////////////////////////////
// User Management Messages
message CreateUserRequest {
Identity identity = 1;
}
message CreateUserResponse {
}
message GetUserRequest {
string username = 1;
}
message GetUserResponse {
Identity identity = 1;
}
message UpdateUserRequest {
string username = 1;
Identity identity = 2;
}
message UpdateUserResponse {
}
message DeleteUserRequest {
string username = 1;
}
message DeleteUserResponse {
}
message ListUsersRequest {
}
message ListUsersResponse {
repeated string usernames = 1;
}
//////////////////////////////////////////////////
// Access Key Management Messages
message CreateAccessKeyRequest {
string username = 1;
Credential credential = 2;
}
message CreateAccessKeyResponse {
}
message DeleteAccessKeyRequest {
string username = 1;
string access_key = 2;
}
message DeleteAccessKeyResponse {
}
message GetUserByAccessKeyRequest {
string access_key = 1;
}
message GetUserByAccessKeyResponse {
Identity identity = 1;
}
message ListAccessKeysRequest {
string username = 1;
}
message ListAccessKeysResponse {
repeated Credential access_keys = 1;
}
// User Policy Management Messages
message PutUserPolicyRequest {
string username = 1;
string policy_name = 2;
string policy_document = 3;
}
message PutUserPolicyResponse {
}
message GetUserPolicyRequest {
string username = 1;
string policy_name = 2;
}
message GetUserPolicyResponse {
string username = 1;
string policy_name = 2;
string policy_document = 3;
}
message DeleteUserPolicyRequest {
string username = 1;
string policy_name = 2;
}
message DeleteUserPolicyResponse {
}
//////////////////////////////////////////////////
message S3ApiConfiguration {
repeated Identity identities = 1;
repeated Account accounts = 2;
repeated ServiceAccount service_accounts = 3;
repeated Policy policies = 4;
}
message Identity {
string name = 1;
repeated Credential credentials = 2;
repeated string actions = 3;
Account account = 4;
bool disabled = 5; // User status: false = enabled (default), true = disabled
repeated string service_account_ids = 6; // IDs of service accounts owned by this user
repeated string policy_names = 7;
}
message Credential {
string access_key = 1;
string secret_key = 2;
string status = 3; // Access key status: "Active" or "Inactive"
}
message Account {
string id = 1;
string display_name = 2;
string email_address = 3;
}
// ServiceAccount represents a service account - special credentials for applications.
// Service accounts are linked to a parent user and can have restricted permissions.
message ServiceAccount {
string id = 1; // Unique identifier (e.g., "sa-xxxxx")
string parent_user = 2; // Parent identity name
string description = 3; // Optional description
Credential credential = 4; // Access key/secret for this service account
repeated string actions = 5; // Allowed actions (subset of parent)
int64 expiration = 6; // Unix timestamp, 0 = no expiration
bool disabled = 7; // Status: false = enabled (default)
int64 created_at = 8; // Creation timestamp
string created_by = 9; // Who created this service account
}
message PutPolicyRequest {
string name = 1;
string content = 2;
}
message PutPolicyResponse {
}
message GetPolicyRequest {
string name = 1;
}
message GetPolicyResponse {
string name = 1;
string content = 2;
}
message ListPoliciesRequest {
}
message ListPoliciesResponse {
repeated Policy policies = 1;
}
message DeletePolicyRequest {
string name = 1;
}
message DeletePolicyResponse {
}
message Policy {
string name = 1;
string content = 2; // JSON content of the policy
}
//////////////////////////////////////////////////
// Service Account Messages
message CreateServiceAccountRequest {
ServiceAccount service_account = 1;
}
message CreateServiceAccountResponse {
}
message UpdateServiceAccountRequest {
string id = 1;
ServiceAccount service_account = 2;
}
message UpdateServiceAccountResponse {
}
message DeleteServiceAccountRequest {
string id = 1;
}
message DeleteServiceAccountResponse {
}
message GetServiceAccountRequest {
string id = 1;
}
message GetServiceAccountResponse {
ServiceAccount service_account = 1;
}
message ListServiceAccountsRequest {
}
message ListServiceAccountsResponse {
repeated ServiceAccount service_accounts = 1;
}
message GetServiceAccountByAccessKeyRequest {
string access_key = 1;
}
message GetServiceAccountByAccessKeyResponse {
ServiceAccount service_account = 1;
}
//////////////////////////////////////////////////
// S3 IAM Cache Management
// Designed for unidirectional propagation from Filer to S3 Servers
message PutIdentityRequest {
Identity identity = 1;
}
message PutIdentityResponse {
}
message RemoveIdentityRequest {
string username = 1;
}
message RemoveIdentityResponse {
}