migrate IAM policies to multi-file storage (#8114)
* Add IAM gRPC service definition - Add GetConfiguration/PutConfiguration for config management - Add CreateUser/GetUser/UpdateUser/DeleteUser/ListUsers for user management - Add CreateAccessKey/DeleteAccessKey/GetUserByAccessKey for access key management - Methods mirror existing IAM HTTP API functionality * Add IAM gRPC handlers on filer server - Implement IamGrpcServer with CredentialManager integration - Handle configuration get/put operations - Handle user CRUD operations - Handle access key create/delete operations - All methods delegate to CredentialManager for actual storage * Wire IAM gRPC service to filer server - Add CredentialManager field to FilerOption and FilerServer - Import credential store implementations in filer command - Initialize CredentialManager from credential.toml if available - Register IAM gRPC service on filer gRPC server - Enable credential management via gRPC alongside existing filer services * Regenerate IAM protobuf with gRPC service methods * fix: compilation error in DeleteUser * fix: address code review comments for IAM migration * feat: migrate policies to multi-file layout and fix identity duplicated content * refactor: remove configuration.json and migrate Service Accounts to multi-file layout * refactor: standardize Service Accounts as distinct store entities and fix Admin Server persistence * config: set ServiceAccountsDirectory to /etc/iam/service_accounts * Fix Chrome dialog auto-dismiss with Bootstrap modals - Add modal-alerts.js library with Bootstrap modal replacements - Replace all 15 confirm() calls with showConfirm/showDeleteConfirm - Auto-override window.alert() for all alert() calls - Fixes Chrome 132+ aggressively blocking native dialogs * Upgrade Bootstrap from 5.3.2 to 5.3.8 * Fix syntax error in object_store_users.templ - remove duplicate closing braces * create policy * display errors * migrate to multi-file policies * address PR feedback: use showDeleteConfirm and showErrorMessage in policies.templ, refine migration check * Update policies_templ.go * add service account to iam grpc * iam: fix potential path traversal in policy names by validating name pattern * iam: add GetServiceAccountByAccessKey to CredentialStore interface * iam: implement service account support for PostgresStore Includes full CRUD operations and efficient lookup by access key. * iam: implement GetServiceAccountByAccessKey for filer_etc, grpc, and memory stores Provides efficient lookup of service accounts by access key where possible, with linear scan fallbacks for file-based stores. * iam: remove filer_multiple support Deleted its implementation and references in imports, scaffold config, and core interface constants. Redundant with filer_etc. * clear comment * dash: robustify service account construction - Guard against nil sa.Credential when constructing responses - Fix Expiration logic to only set if > 0, avoiding Unix epoch 1970 - Ensure consistency across Get, Create, and Update handlers * credential/filer_etc: improve error propagation in configuration handlers - Return error from loadServiceAccountsFromMultiFile to callers - Ensure listEntries errors in SaveConfiguration (cleanup logic) are propagated unless they are "not found" failures. - Fixes potential silent failures during IAM configuration sync. * credential/filer_etc: add existence check to CreateServiceAccount Ensures consistency with other stores by preventing accidental overwrite of existing service accounts during creation. * credential/memory: improve store robustness and Reset logic - Enforce ID immutability in UpdateServiceAccount to prevent orphans - Update Reset() to also clear the policies map, ensuring full state cleanup for tests. * dash: improve service account robustness and policy docs - Wrap parent user lookup errors to preserve context - Strictly validate Status field in UpdateServiceAccount - Add deprecation comments to legacy policy management methods * credential/filer_etc: protect against path traversal in service accounts Implemented ID validation (alphanumeric, underscores, hyphens) and applied it to Get, Save, and Delete operations to ensure no directory traversal via saId.json filenames. * credential/postgres: improve robustness and cleanup comments - Removed brainstorming comments in GetServiceAccountByAccessKey - Added missing rows.Err() check during iteration - Properly propagate Scan and Unmarshal errors instead of swallowing them * admin: unify UI alerts and confirmations using Bootstrap modals - Updated modal-alerts.js with improved automated alert type detection - Replaced native alert() and confirm() with showAlert(), showConfirm(), and showDeleteConfirm() across various Templ components - Improved UX for delete operations by providing better context and styling - Ensured consistent error reporting across IAM and Maintenance views * admin: additional UI consistency fixes for alerts and confirmations - Replaced native alert() and confirm() with Bootstrap modals in: - EC volumes (repair flow) - Collection details (repair flow) - File browser (properties and delete) - Maintenance config schema (save and reset) - Improved delete confirmation in file browser with item context - Ensured consistent success/error/info styling for all feedbacks * make * iam: add GetServiceAccountByAccessKey RPC and update GetConfiguration * iam: implement GetServiceAccountByAccessKey on server and client * iam: centralize policy and service account validation * iam: optimize MemoryStore service account lookups with indexing * iam: fix postgres service_accounts table and optimize lookups * admin: refactor modal alerts and clean up dashboard logic * admin: fix EC shards table layout mismatch * admin: URL-encode IAM path parameters for safety * admin: implement pauseWorker logic in maintenance view * iam: add rows.Err() check to postgres ListServiceAccounts * iam: standardize ErrServiceAccountNotFound across credential stores * iam: map ErrServiceAccountNotFound to codes.NotFound in DeleteServiceAccount * iam: refine service account store logic, errors and schema * iam: add validation to GetServiceAccountByAccessKey * admin: refine modal titles and ensure URL safety * admin: address bot review comments for alerts and async usage * iam: fix syntax error by restoring missing function declaration * [FilerEtcStore] improve error handling in CreateServiceAccount Refine error handling to provide clearer messages when checking for existing service accounts. * [PostgresStore] add nil guards and validation to service account methods Ensure input parameters are not nil and required IDs are present to prevent runtime panics and ensure data integrity. * [JS] add shared IAM utility script Consolidate common IAM operations like deleteUser and deleteAccessKey into a shared utility script for better maintainability. * [View] include shared IAM utilities in layout Include iam-utils.js in the main layout to make IAM functions available across all administrative pages. * [View] refactor IAM logic and restore async in EC Shards view Remove redundant local IAM functions and ensure that delete confirmation callbacks are properly marked as async. * [View] consolidate IAM logic in Object Store Users view Remove redundant local definitions of deleteUser and deleteAccessKey, relying on the shared utilities instead. * [View] update generated templ files for UI consistency * credential/postgres: remove redundant name column from service_accounts table The id is already used as the unique identifier and was being copied to the name column. This removes the name column from the schema and updates the INSERT/UPDATE queries. * credential/filer_etc: improve logging for policy migration failures Added Errorf log if AtomicRenameEntry fails during migration to ensure visibility of common failure points. * credential: allow uppercase characters in service account ID username Updated ServiceAccountIdPattern to allow [A-Za-z0-9_-]+ for the username component, matching the actual service account creation logic which uses the parent user name directly. * Update object_store_users_templ.go * admin: fix ec_shards pagination to handle numeric page arguments Updated goToPage in cluster_ec_shards.templ to accept either an Event or a numeric page argument. This prevents errors when goToPage(1) is called directly. Corrected both the .templ source and generated Go code. * credential/filer_etc: improve service account storage robustness Added nil guard to saveServiceAccount, updated GetServiceAccount to return ErrServiceAccountNotFound for empty data, and improved deleteServiceAccount to handle response-level Filer errors.
This commit is contained in:
@@ -125,6 +125,26 @@ func (cm *CredentialManager) GetPolicy(ctx context.Context, name string) (*polic
|
||||
return cm.store.GetPolicy(ctx, name)
|
||||
}
|
||||
|
||||
// CreatePolicy creates a new policy (if supported by the store)
|
||||
func (cm *CredentialManager) CreatePolicy(ctx context.Context, name string, document policy_engine.PolicyDocument) error {
|
||||
// Check if the store implements PolicyManager interface with CreatePolicy
|
||||
if policyStore, ok := cm.store.(PolicyManager); ok {
|
||||
return policyStore.CreatePolicy(ctx, name, document)
|
||||
}
|
||||
// Fallback to PutPolicy for stores that only implement CredentialStore
|
||||
return cm.store.PutPolicy(ctx, name, document)
|
||||
}
|
||||
|
||||
// UpdatePolicy updates an existing policy (if supported by the store)
|
||||
func (cm *CredentialManager) UpdatePolicy(ctx context.Context, name string, document policy_engine.PolicyDocument) error {
|
||||
// Check if the store implements PolicyManager interface with UpdatePolicy
|
||||
if policyStore, ok := cm.store.(PolicyManager); ok {
|
||||
return policyStore.UpdatePolicy(ctx, name, document)
|
||||
}
|
||||
// Fallback to PutPolicy for stores that only implement CredentialStore
|
||||
return cm.store.PutPolicy(ctx, name, document)
|
||||
}
|
||||
|
||||
// Shutdown performs cleanup
|
||||
func (cm *CredentialManager) Shutdown() {
|
||||
if cm.store != nil {
|
||||
@@ -152,3 +172,28 @@ func GetAvailableStores() []CredentialStoreTypeName {
|
||||
}
|
||||
return storeNames
|
||||
}
|
||||
|
||||
// CreateServiceAccount creates a new service account
|
||||
func (cm *CredentialManager) CreateServiceAccount(ctx context.Context, sa *iam_pb.ServiceAccount) error {
|
||||
return cm.store.CreateServiceAccount(ctx, sa)
|
||||
}
|
||||
|
||||
// UpdateServiceAccount updates an existing service account
|
||||
func (cm *CredentialManager) UpdateServiceAccount(ctx context.Context, id string, sa *iam_pb.ServiceAccount) error {
|
||||
return cm.store.UpdateServiceAccount(ctx, id, sa)
|
||||
}
|
||||
|
||||
// DeleteServiceAccount removes a service account
|
||||
func (cm *CredentialManager) DeleteServiceAccount(ctx context.Context, id string) error {
|
||||
return cm.store.DeleteServiceAccount(ctx, id)
|
||||
}
|
||||
|
||||
// GetServiceAccount retrieves a service account by ID
|
||||
func (cm *CredentialManager) GetServiceAccount(ctx context.Context, id string) (*iam_pb.ServiceAccount, error) {
|
||||
return cm.store.GetServiceAccount(ctx, id)
|
||||
}
|
||||
|
||||
// ListServiceAccounts returns all service accounts
|
||||
func (cm *CredentialManager) ListServiceAccounts(ctx context.Context) ([]*iam_pb.ServiceAccount, error) {
|
||||
return cm.store.ListServiceAccounts(ctx)
|
||||
}
|
||||
|
||||
@@ -11,9 +11,10 @@ import (
|
||||
)
|
||||
|
||||
var (
|
||||
ErrUserNotFound = errors.New("user not found")
|
||||
ErrUserAlreadyExists = errors.New("user already exists")
|
||||
ErrAccessKeyNotFound = errors.New("access key not found")
|
||||
ErrUserNotFound = errors.New("user not found")
|
||||
ErrUserAlreadyExists = errors.New("user already exists")
|
||||
ErrAccessKeyNotFound = errors.New("access key not found")
|
||||
ErrServiceAccountNotFound = errors.New("service account not found")
|
||||
)
|
||||
|
||||
// CredentialStoreTypeName represents the type name of a credential store
|
||||
@@ -21,11 +22,10 @@ type CredentialStoreTypeName string
|
||||
|
||||
// Credential store name constants
|
||||
const (
|
||||
StoreTypeMemory CredentialStoreTypeName = "memory"
|
||||
StoreTypeFilerEtc CredentialStoreTypeName = "filer_etc"
|
||||
StoreTypeFilerMultiple CredentialStoreTypeName = "filer_multiple"
|
||||
StoreTypePostgres CredentialStoreTypeName = "postgres"
|
||||
StoreTypeGrpc CredentialStoreTypeName = "grpc"
|
||||
StoreTypeMemory CredentialStoreTypeName = "memory"
|
||||
StoreTypeFilerEtc CredentialStoreTypeName = "filer_etc"
|
||||
StoreTypePostgres CredentialStoreTypeName = "postgres"
|
||||
StoreTypeGrpc CredentialStoreTypeName = "grpc"
|
||||
)
|
||||
|
||||
// CredentialStore defines the interface for user credential storage and retrieval
|
||||
@@ -73,6 +73,14 @@ type CredentialStore interface {
|
||||
DeletePolicy(ctx context.Context, name string) error
|
||||
GetPolicy(ctx context.Context, name string) (*policy_engine.PolicyDocument, error)
|
||||
|
||||
// Service Account Management
|
||||
CreateServiceAccount(ctx context.Context, sa *iam_pb.ServiceAccount) error
|
||||
UpdateServiceAccount(ctx context.Context, id string, sa *iam_pb.ServiceAccount) error
|
||||
DeleteServiceAccount(ctx context.Context, id string) error
|
||||
GetServiceAccount(ctx context.Context, id string) (*iam_pb.ServiceAccount, error)
|
||||
ListServiceAccounts(ctx context.Context) ([]*iam_pb.ServiceAccount, error)
|
||||
GetServiceAccountByAccessKey(ctx context.Context, accessKey string) (*iam_pb.ServiceAccount, error)
|
||||
|
||||
// Shutdown performs cleanup when the store is being shut down
|
||||
Shutdown()
|
||||
}
|
||||
|
||||
@@ -1,10 +1,10 @@
|
||||
package filer_etc
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"strings"
|
||||
|
||||
"github.com/seaweedfs/seaweedfs/weed/credential"
|
||||
"github.com/seaweedfs/seaweedfs/weed/filer"
|
||||
@@ -14,10 +14,10 @@ import (
|
||||
)
|
||||
|
||||
const (
|
||||
IamIdentitiesDirectory = "identities"
|
||||
IamConfigurationFile = "configuration.json"
|
||||
IamLegacyIdentityFile = "identity.json"
|
||||
IamLegacyIdentityOldFile = "identity.json.old"
|
||||
IamIdentitiesDirectory = "identities"
|
||||
IamServiceAccountsDirectory = "service_accounts"
|
||||
IamLegacyIdentityFile = "identity.json"
|
||||
IamLegacyIdentityOldFile = "identity.json.old"
|
||||
)
|
||||
|
||||
func (store *FilerEtcStore) LoadConfiguration(ctx context.Context) (*iam_pb.S3ApiConfiguration, error) {
|
||||
@@ -35,13 +35,17 @@ func (store *FilerEtcStore) LoadConfiguration(ctx context.Context) (*iam_pb.S3Ap
|
||||
}
|
||||
}
|
||||
|
||||
// 2. Load from multi-file structure (high priority, overrides legacy)
|
||||
// This will merge identities into s3cfg
|
||||
// 2. Load from multi-file structure (high priority, overrides legacy details)
|
||||
if _, err := store.loadFromMultiFile(ctx, s3cfg); err != nil {
|
||||
return s3cfg, err
|
||||
}
|
||||
|
||||
// 3. Perform migration if we loaded legacy config
|
||||
// 3. Load service accounts
|
||||
if err := store.loadServiceAccountsFromMultiFile(ctx, s3cfg); err != nil {
|
||||
return s3cfg, fmt.Errorf("failed to load service accounts: %w", err)
|
||||
}
|
||||
|
||||
// 4. Perform migration if we loaded legacy config
|
||||
// This ensures that all identities (including legacy ones) are written to individual files
|
||||
// and the legacy file is renamed.
|
||||
if foundLegacy {
|
||||
@@ -72,8 +76,11 @@ func (store *FilerEtcStore) loadFromMultiFile(ctx context.Context, s3cfg *iam_pb
|
||||
dir := filer.IamConfigDirectory + "/" + IamIdentitiesDirectory
|
||||
entries, err := listEntries(ctx, client, dir)
|
||||
if err != nil {
|
||||
// If directory doesn't exist, it's not multi-file yet
|
||||
return nil
|
||||
if err == filer_pb.ErrNotFound {
|
||||
// If directory doesn't exist, it's not multi-file yet
|
||||
return nil
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
for _, entry := range entries {
|
||||
@@ -117,20 +124,6 @@ func (store *FilerEtcStore) loadFromMultiFile(ctx context.Context, s3cfg *iam_pb
|
||||
return false, err
|
||||
}
|
||||
|
||||
// 2. Load configuration.json (Accounts, etc.)
|
||||
content, found, err := store.readInsideFiler(filer.IamConfigDirectory, IamConfigurationFile)
|
||||
if err != nil {
|
||||
return false, err
|
||||
}
|
||||
if found && len(content) > 0 {
|
||||
tempCfg := &iam_pb.S3ApiConfiguration{}
|
||||
if err := filer.ParseS3ConfigurationFromBytes(content, tempCfg); err == nil {
|
||||
// Overwrite accounts from configuration.json (high priority)
|
||||
s3cfg.Accounts = tempCfg.Accounts
|
||||
}
|
||||
return true, nil
|
||||
}
|
||||
|
||||
return hasIdentities, nil
|
||||
}
|
||||
|
||||
@@ -144,27 +137,16 @@ func (store *FilerEtcStore) migrateToMultiFile(ctx context.Context, s3cfg *iam_p
|
||||
}
|
||||
}
|
||||
|
||||
// 2. Save rest of configuration
|
||||
if err := store.withFilerClient(func(client filer_pb.SeaweedFilerClient) error {
|
||||
// Create config with only accounts
|
||||
cleanCfg := &iam_pb.S3ApiConfiguration{
|
||||
Accounts: s3cfg.Accounts,
|
||||
}
|
||||
var buf bytes.Buffer
|
||||
if err := filer.ProtoToText(&buf, cleanCfg); err != nil {
|
||||
// 2. Save all service accounts
|
||||
for _, sa := range s3cfg.ServiceAccounts {
|
||||
if err := store.saveServiceAccount(ctx, sa); err != nil {
|
||||
return err
|
||||
}
|
||||
return filer.SaveInsideFiler(client, filer.IamConfigDirectory, IamConfigurationFile, buf.Bytes())
|
||||
}); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
// 3. Rename legacy file
|
||||
return store.withFilerClient(func(client filer_pb.SeaweedFilerClient) error {
|
||||
// We use atomic rename if possible, but Filer 'AtomicRenameEntry' exists in filer_pb
|
||||
// util.JoinPath(filer.IamConfigDirectory, IamLegacyIdentityFile)
|
||||
|
||||
_, err := client.AtomicRenameEntry(context.Background(), &filer_pb.AtomicRenameEntryRequest{
|
||||
_, err := client.AtomicRenameEntry(ctx, &filer_pb.AtomicRenameEntryRequest{
|
||||
OldDirectory: filer.IamConfigDirectory,
|
||||
OldName: IamLegacyIdentityFile,
|
||||
NewDirectory: filer.IamConfigDirectory,
|
||||
@@ -182,31 +164,22 @@ func (store *FilerEtcStore) SaveConfiguration(ctx context.Context, config *iam_p
|
||||
}
|
||||
}
|
||||
|
||||
// 2. Save configuration file (accounts)
|
||||
err := store.withFilerClient(func(client filer_pb.SeaweedFilerClient) error {
|
||||
cleanCfg := &iam_pb.S3ApiConfiguration{
|
||||
Accounts: config.Accounts,
|
||||
}
|
||||
var buf bytes.Buffer
|
||||
if err := filer.ProtoToText(&buf, cleanCfg); err != nil {
|
||||
// 2. Save all service accounts
|
||||
for _, sa := range config.ServiceAccounts {
|
||||
if err := store.saveServiceAccount(ctx, sa); err != nil {
|
||||
return err
|
||||
}
|
||||
return filer.SaveInsideFiler(client, filer.IamConfigDirectory, IamConfigurationFile, buf.Bytes())
|
||||
})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
// 3. Cleanup removed identities (Full Sync)
|
||||
// Get list of existing identity files
|
||||
// Compare with config.Identities
|
||||
// Delete unknown ones
|
||||
|
||||
return store.withFilerClient(func(client filer_pb.SeaweedFilerClient) error {
|
||||
if err := store.withFilerClient(func(client filer_pb.SeaweedFilerClient) error {
|
||||
dir := filer.IamConfigDirectory + "/" + IamIdentitiesDirectory
|
||||
entries, err := listEntries(ctx, client, dir)
|
||||
if err != nil {
|
||||
return nil // Should exist by now
|
||||
if err == filer_pb.ErrNotFound {
|
||||
return nil
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
validNames := make(map[string]bool)
|
||||
@@ -217,7 +190,7 @@ func (store *FilerEtcStore) SaveConfiguration(ctx context.Context, config *iam_p
|
||||
for _, entry := range entries {
|
||||
if !entry.IsDirectory && !validNames[entry.Name] {
|
||||
// Delete obsolete identity file
|
||||
if _, err := client.DeleteEntry(context.Background(), &filer_pb.DeleteEntryRequest{
|
||||
if _, err := client.DeleteEntry(ctx, &filer_pb.DeleteEntryRequest{
|
||||
Directory: dir,
|
||||
Name: entry.Name,
|
||||
}); err != nil {
|
||||
@@ -226,7 +199,42 @@ func (store *FilerEtcStore) SaveConfiguration(ctx context.Context, config *iam_p
|
||||
}
|
||||
}
|
||||
return nil
|
||||
})
|
||||
}); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
// 4. Cleanup removed service accounts (Full Sync)
|
||||
if err := store.withFilerClient(func(client filer_pb.SeaweedFilerClient) error {
|
||||
dir := filer.IamConfigDirectory + "/" + IamServiceAccountsDirectory
|
||||
entries, err := listEntries(ctx, client, dir)
|
||||
if err != nil {
|
||||
if err == filer_pb.ErrNotFound {
|
||||
return nil
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
validNames := make(map[string]bool)
|
||||
for _, sa := range config.ServiceAccounts {
|
||||
validNames[sa.Id+".json"] = true
|
||||
}
|
||||
|
||||
for _, entry := range entries {
|
||||
if !entry.IsDirectory && !validNames[entry.Name] {
|
||||
if _, err := client.DeleteEntry(ctx, &filer_pb.DeleteEntryRequest{
|
||||
Directory: dir,
|
||||
Name: entry.Name,
|
||||
}); err != nil {
|
||||
glog.Warningf("Failed to delete obsolete service account file %s: %v", entry.Name, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
func (store *FilerEtcStore) CreateUser(ctx context.Context, identity *iam_pb.Identity) error {
|
||||
@@ -274,16 +282,20 @@ func (store *FilerEtcStore) UpdateUser(ctx context.Context, username string, ide
|
||||
}
|
||||
|
||||
func (store *FilerEtcStore) DeleteUser(ctx context.Context, username string) error {
|
||||
// Verify existence first to return ErrUserNotFound if applicable
|
||||
if _, err := store.GetUser(ctx, username); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
return store.withFilerClient(func(client filer_pb.SeaweedFilerClient) error {
|
||||
_, err := client.DeleteEntry(context.Background(), &filer_pb.DeleteEntryRequest{
|
||||
_, err := client.DeleteEntry(ctx, &filer_pb.DeleteEntryRequest{
|
||||
Directory: filer.IamConfigDirectory + "/" + IamIdentitiesDirectory,
|
||||
Name: username + ".json",
|
||||
})
|
||||
if err != nil {
|
||||
// Map specific gRPC error to ErrUserNotFound if possible, but DeleteEntry usually returns success even if not found
|
||||
// unless strict. 'credential.ErrUserNotFound' is expected by caller?
|
||||
// The caller `DeleteUser` in handlers usually explicitly checks `ErrUserNotFound`.
|
||||
// Ideally we verify existence first?
|
||||
if strings.Contains(err.Error(), filer_pb.ErrNotFound.Error()) {
|
||||
return credential.ErrUserNotFound
|
||||
}
|
||||
return err
|
||||
}
|
||||
return nil
|
||||
@@ -295,6 +307,9 @@ func (store *FilerEtcStore) ListUsers(ctx context.Context) ([]string, error) {
|
||||
err := store.withFilerClient(func(client filer_pb.SeaweedFilerClient) error {
|
||||
entries, err := listEntries(ctx, client, filer.IamConfigDirectory+"/"+IamIdentitiesDirectory)
|
||||
if err != nil {
|
||||
if err == filer_pb.ErrNotFound {
|
||||
return nil
|
||||
}
|
||||
return err
|
||||
}
|
||||
for _, entry := range entries {
|
||||
@@ -307,29 +322,66 @@ func (store *FilerEtcStore) ListUsers(ctx context.Context) ([]string, error) {
|
||||
return usernames, err
|
||||
}
|
||||
|
||||
// Access Key methods still need to operate on the identity object
|
||||
// We can reuse GetUser / UpdateUser logic to avoid duplicating file IO code here,
|
||||
// or implement optimized read-modify-write.
|
||||
// Reusing GetUser/saveIdentity is cleanest.
|
||||
|
||||
func (store *FilerEtcStore) GetUserByAccessKey(ctx context.Context, accessKey string) (*iam_pb.Identity, error) {
|
||||
// This is inefficient in multi-file: requires scanning all files.
|
||||
// Assuming number of users is not huge.
|
||||
// For huge number of users, we'd need an index.
|
||||
// Optimized: Iterate over identity files directly instead of loading full config.
|
||||
// This avoids triggering migration side effects.
|
||||
|
||||
var foundIdentity *iam_pb.Identity
|
||||
|
||||
err := store.withFilerClient(func(client filer_pb.SeaweedFilerClient) error {
|
||||
dir := filer.IamConfigDirectory + "/" + IamIdentitiesDirectory
|
||||
entries, err := listEntries(ctx, client, dir)
|
||||
if err != nil {
|
||||
// If not found, check legacy file? No, optimization requested to avoid side effects.
|
||||
// If migration hasn't run, this will return empty/not found.
|
||||
if err == filer_pb.ErrNotFound {
|
||||
return nil
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
for _, entry := range entries {
|
||||
if entry.IsDirectory || !strings.HasSuffix(entry.Name, ".json") {
|
||||
continue
|
||||
}
|
||||
|
||||
// Read file content
|
||||
var content []byte
|
||||
if len(entry.Content) > 0 {
|
||||
content = entry.Content
|
||||
} else {
|
||||
c, err := filer.ReadInsideFiler(client, dir, entry.Name)
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
content = c
|
||||
}
|
||||
|
||||
if len(content) > 0 {
|
||||
identity := &iam_pb.Identity{}
|
||||
if err := json.Unmarshal(content, identity); err != nil {
|
||||
continue
|
||||
}
|
||||
|
||||
for _, cred := range identity.Credentials {
|
||||
if cred.AccessKey == accessKey {
|
||||
foundIdentity = identity
|
||||
return nil // Found match, stop iteration
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
return nil
|
||||
})
|
||||
|
||||
s3cfg, err := store.LoadConfiguration(ctx)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
for _, identity := range s3cfg.Identities {
|
||||
for _, credential := range identity.Credentials {
|
||||
if credential.AccessKey == accessKey {
|
||||
// Return the specific identity
|
||||
return identity, nil
|
||||
}
|
||||
}
|
||||
if foundIdentity != nil {
|
||||
return foundIdentity, nil
|
||||
}
|
||||
|
||||
return nil, credential.ErrAccessKeyNotFound
|
||||
}
|
||||
|
||||
|
||||
@@ -3,22 +3,31 @@ package filer_etc
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"strings"
|
||||
|
||||
"github.com/seaweedfs/seaweedfs/weed/credential"
|
||||
"github.com/seaweedfs/seaweedfs/weed/filer"
|
||||
"github.com/seaweedfs/seaweedfs/weed/glog"
|
||||
"github.com/seaweedfs/seaweedfs/weed/pb/filer_pb"
|
||||
"github.com/seaweedfs/seaweedfs/weed/s3api/policy_engine"
|
||||
)
|
||||
|
||||
const (
|
||||
IamPoliciesDirectory = "policies"
|
||||
IamLegacyPoliciesOldFile = "policies.json.old"
|
||||
)
|
||||
|
||||
type PoliciesCollection struct {
|
||||
Policies map[string]policy_engine.PolicyDocument `json:"policies"`
|
||||
}
|
||||
|
||||
func validatePolicyName(name string) error {
|
||||
return credential.ValidatePolicyName(name)
|
||||
}
|
||||
|
||||
// GetPolicies retrieves all IAM policies from the filer
|
||||
func (store *FilerEtcStore) GetPolicies(ctx context.Context) (map[string]policy_engine.PolicyDocument, error) {
|
||||
policiesCollection := &PoliciesCollection{
|
||||
Policies: make(map[string]policy_engine.PolicyDocument),
|
||||
}
|
||||
policies := make(map[string]policy_engine.PolicyDocument)
|
||||
|
||||
// Check if filer client is configured (with mutex protection)
|
||||
store.mu.RLock()
|
||||
@@ -27,75 +36,142 @@ func (store *FilerEtcStore) GetPolicies(ctx context.Context) (map[string]policy_
|
||||
|
||||
if !configured {
|
||||
glog.V(1).Infof("Filer client not configured for policy retrieval, returning empty policies")
|
||||
// Return empty policies if filer client is not configured
|
||||
return policiesCollection.Policies, nil
|
||||
return policies, nil
|
||||
}
|
||||
|
||||
glog.V(2).Infof("Loading IAM policies from %s/%s (using current active filer)",
|
||||
filer.IamConfigDirectory, filer.IamPoliciesFile)
|
||||
|
||||
err := store.withFilerClient(func(client filer_pb.SeaweedFilerClient) error {
|
||||
// Use ReadInsideFiler instead of ReadEntry since policies.json is small
|
||||
// and stored inline. ReadEntry requires a master client for chunked files,
|
||||
// but ReadInsideFiler only reads inline content.
|
||||
content, err := filer.ReadInsideFiler(client, filer.IamConfigDirectory, filer.IamPoliciesFile)
|
||||
if err != nil {
|
||||
if err == filer_pb.ErrNotFound {
|
||||
glog.V(1).Infof("Policies file not found at %s/%s, returning empty policies",
|
||||
filer.IamConfigDirectory, filer.IamPoliciesFile)
|
||||
// If file doesn't exist, return empty collection
|
||||
return nil
|
||||
}
|
||||
glog.Errorf("Failed to read IAM policies file from %s/%s: %v",
|
||||
filer.IamConfigDirectory, filer.IamPoliciesFile, err)
|
||||
return err
|
||||
}
|
||||
|
||||
if len(content) == 0 {
|
||||
glog.V(2).Infof("IAM policies file at %s/%s is empty",
|
||||
filer.IamConfigDirectory, filer.IamPoliciesFile)
|
||||
return nil
|
||||
}
|
||||
|
||||
glog.V(2).Infof("Read %d bytes from %s/%s",
|
||||
len(content), filer.IamConfigDirectory, filer.IamPoliciesFile)
|
||||
|
||||
if err := json.Unmarshal(content, policiesCollection); err != nil {
|
||||
glog.Errorf("Failed to parse IAM policies from %s/%s: %v",
|
||||
filer.IamConfigDirectory, filer.IamPoliciesFile, err)
|
||||
return err
|
||||
}
|
||||
|
||||
glog.V(1).Infof("Successfully loaded %d IAM policies", len(policiesCollection.Policies))
|
||||
return nil
|
||||
})
|
||||
|
||||
// 1. Load from legacy single file (low priority)
|
||||
content, foundLegacy, err := store.readInsideFiler(filer.IamConfigDirectory, filer.IamPoliciesFile)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// Log policy names for debugging
|
||||
if glog.V(2) && len(policiesCollection.Policies) > 0 {
|
||||
for policyName := range policiesCollection.Policies {
|
||||
glog.V(2).Infof(" Policy: %s", policyName)
|
||||
if foundLegacy && len(content) > 0 {
|
||||
policiesCollection := &PoliciesCollection{
|
||||
Policies: make(map[string]policy_engine.PolicyDocument),
|
||||
}
|
||||
if err := json.Unmarshal(content, policiesCollection); err != nil {
|
||||
glog.Errorf("Failed to parse legacy IAM policies from %s/%s: %v",
|
||||
filer.IamConfigDirectory, filer.IamPoliciesFile, err)
|
||||
} else {
|
||||
for name, policy := range policiesCollection.Policies {
|
||||
policies[name] = policy
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return policiesCollection.Policies, nil
|
||||
// 2. Load from multi-file structure (high priority, overrides legacy)
|
||||
if err := store.loadPoliciesFromMultiFile(ctx, policies); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// 3. Perform migration if we loaded legacy config
|
||||
if foundLegacy {
|
||||
if err := store.migratePoliciesToMultiFile(ctx, policies); err != nil {
|
||||
glog.Errorf("Failed to migrate IAM policies to multi-file layout: %v", err)
|
||||
return policies, err
|
||||
}
|
||||
}
|
||||
|
||||
return policies, nil
|
||||
}
|
||||
|
||||
func (store *FilerEtcStore) loadPoliciesFromMultiFile(ctx context.Context, policies map[string]policy_engine.PolicyDocument) error {
|
||||
return store.withFilerClient(func(client filer_pb.SeaweedFilerClient) error {
|
||||
dir := filer.IamConfigDirectory + "/" + IamPoliciesDirectory
|
||||
entries, err := listEntries(ctx, client, dir)
|
||||
if err != nil {
|
||||
if err == filer_pb.ErrNotFound {
|
||||
return nil
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
for _, entry := range entries {
|
||||
if entry.IsDirectory {
|
||||
continue
|
||||
}
|
||||
|
||||
var content []byte
|
||||
if len(entry.Content) > 0 {
|
||||
content = entry.Content
|
||||
} else {
|
||||
c, err := filer.ReadInsideFiler(client, dir, entry.Name)
|
||||
if err != nil {
|
||||
glog.Warningf("Failed to read policy file %s: %v", entry.Name, err)
|
||||
continue
|
||||
}
|
||||
content = c
|
||||
}
|
||||
|
||||
if len(content) > 0 {
|
||||
var policy policy_engine.PolicyDocument
|
||||
if err := json.Unmarshal(content, &policy); err != nil {
|
||||
glog.Warningf("Failed to unmarshal policy %s: %v", entry.Name, err)
|
||||
continue
|
||||
}
|
||||
|
||||
// The file name is "policyName.json"
|
||||
policyName := entry.Name
|
||||
if len(policyName) > 5 && policyName[len(policyName)-5:] == ".json" {
|
||||
policyName = policyName[:len(policyName)-5]
|
||||
policies[policyName] = policy
|
||||
}
|
||||
}
|
||||
}
|
||||
return nil
|
||||
})
|
||||
}
|
||||
|
||||
func (store *FilerEtcStore) migratePoliciesToMultiFile(ctx context.Context, policies map[string]policy_engine.PolicyDocument) error {
|
||||
glog.Infof("Migrating IAM policies to multi-file layout...")
|
||||
|
||||
// 1. Save all policies to individual files
|
||||
for name, policy := range policies {
|
||||
if err := store.savePolicy(ctx, name, policy); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
|
||||
// 2. Rename legacy file
|
||||
return store.withFilerClient(func(client filer_pb.SeaweedFilerClient) error {
|
||||
_, err := client.AtomicRenameEntry(ctx, &filer_pb.AtomicRenameEntryRequest{
|
||||
OldDirectory: filer.IamConfigDirectory,
|
||||
OldName: filer.IamPoliciesFile,
|
||||
NewDirectory: filer.IamConfigDirectory,
|
||||
NewName: IamLegacyPoliciesOldFile,
|
||||
})
|
||||
if err != nil {
|
||||
glog.Errorf("Failed to rename legacy IAM policies file %s/%s to %s: %v",
|
||||
filer.IamConfigDirectory, filer.IamPoliciesFile, IamLegacyPoliciesOldFile, err)
|
||||
}
|
||||
return err
|
||||
})
|
||||
}
|
||||
|
||||
func (store *FilerEtcStore) savePolicy(ctx context.Context, name string, document policy_engine.PolicyDocument) error {
|
||||
if err := validatePolicyName(name); err != nil {
|
||||
return err
|
||||
}
|
||||
return store.withFilerClient(func(client filer_pb.SeaweedFilerClient) error {
|
||||
data, err := json.Marshal(document)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return filer.SaveInsideFiler(client, filer.IamConfigDirectory+"/"+IamPoliciesDirectory, name+".json", data)
|
||||
})
|
||||
}
|
||||
|
||||
// CreatePolicy creates a new IAM policy in the filer
|
||||
func (store *FilerEtcStore) CreatePolicy(ctx context.Context, name string, document policy_engine.PolicyDocument) error {
|
||||
return store.updatePolicies(ctx, func(policies map[string]policy_engine.PolicyDocument) {
|
||||
policies[name] = document
|
||||
})
|
||||
return store.savePolicy(ctx, name, document)
|
||||
}
|
||||
|
||||
// UpdatePolicy updates an existing IAM policy in the filer
|
||||
func (store *FilerEtcStore) UpdatePolicy(ctx context.Context, name string, document policy_engine.PolicyDocument) error {
|
||||
return store.updatePolicies(ctx, func(policies map[string]policy_engine.PolicyDocument) {
|
||||
policies[name] = document
|
||||
})
|
||||
return store.savePolicy(ctx, name, document)
|
||||
}
|
||||
|
||||
// PutPolicy creates or updates an IAM policy in the filer
|
||||
@@ -105,46 +181,56 @@ func (store *FilerEtcStore) PutPolicy(ctx context.Context, name string, document
|
||||
|
||||
// DeletePolicy deletes an IAM policy from the filer
|
||||
func (store *FilerEtcStore) DeletePolicy(ctx context.Context, name string) error {
|
||||
return store.updatePolicies(ctx, func(policies map[string]policy_engine.PolicyDocument) {
|
||||
delete(policies, name)
|
||||
})
|
||||
}
|
||||
|
||||
// updatePolicies is a helper method to update policies atomically
|
||||
func (store *FilerEtcStore) updatePolicies(ctx context.Context, updateFunc func(map[string]policy_engine.PolicyDocument)) error {
|
||||
// Load existing policies
|
||||
policies, err := store.GetPolicies(ctx)
|
||||
if err != nil {
|
||||
if err := validatePolicyName(name); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
// Apply update
|
||||
updateFunc(policies)
|
||||
|
||||
// Save back to filer
|
||||
policiesCollection := &PoliciesCollection{
|
||||
Policies: policies,
|
||||
}
|
||||
|
||||
data, err := json.Marshal(policiesCollection)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
return store.withFilerClient(func(client filer_pb.SeaweedFilerClient) error {
|
||||
return filer.SaveInsideFiler(client, filer.IamConfigDirectory, filer.IamPoliciesFile, data)
|
||||
_, err := client.DeleteEntry(ctx, &filer_pb.DeleteEntryRequest{
|
||||
Directory: filer.IamConfigDirectory + "/" + IamPoliciesDirectory,
|
||||
Name: name + ".json",
|
||||
})
|
||||
if err != nil && !strings.Contains(err.Error(), filer_pb.ErrNotFound.Error()) {
|
||||
return err
|
||||
}
|
||||
return nil
|
||||
})
|
||||
}
|
||||
|
||||
// GetPolicy retrieves a specific IAM policy by name from the filer
|
||||
func (store *FilerEtcStore) GetPolicy(ctx context.Context, name string) (*policy_engine.PolicyDocument, error) {
|
||||
if err := validatePolicyName(name); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
var policy *policy_engine.PolicyDocument
|
||||
err := store.withFilerClient(func(client filer_pb.SeaweedFilerClient) error {
|
||||
data, err := filer.ReadInsideFiler(client, filer.IamConfigDirectory+"/"+IamPoliciesDirectory, name+".json")
|
||||
if err != nil {
|
||||
if err == filer_pb.ErrNotFound {
|
||||
return nil
|
||||
}
|
||||
return err
|
||||
}
|
||||
if len(data) == 0 {
|
||||
return nil
|
||||
}
|
||||
policy = &policy_engine.PolicyDocument{}
|
||||
return json.Unmarshal(data, policy)
|
||||
})
|
||||
|
||||
if policy != nil {
|
||||
return policy, err
|
||||
}
|
||||
|
||||
// fallback to full list if single file read fails (e.g. before migration completes or if partially migrated)
|
||||
// Although migration should happen on first GetPolicies call.
|
||||
policies, err := store.GetPolicies(ctx)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if policy, exists := policies[name]; exists {
|
||||
return &policy, nil
|
||||
if p, exists := policies[name]; exists {
|
||||
return &p, nil
|
||||
}
|
||||
|
||||
return nil, nil // Policy not found
|
||||
|
||||
206
weed/credential/filer_etc/filer_etc_service_account.go
Normal file
206
weed/credential/filer_etc/filer_etc_service_account.go
Normal file
@@ -0,0 +1,206 @@
|
||||
package filer_etc
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"strings"
|
||||
|
||||
"github.com/seaweedfs/seaweedfs/weed/credential"
|
||||
"github.com/seaweedfs/seaweedfs/weed/filer"
|
||||
"github.com/seaweedfs/seaweedfs/weed/glog"
|
||||
"github.com/seaweedfs/seaweedfs/weed/pb/filer_pb"
|
||||
"github.com/seaweedfs/seaweedfs/weed/pb/iam_pb"
|
||||
)
|
||||
|
||||
func validateServiceAccountId(id string) error {
|
||||
return credential.ValidateServiceAccountId(id)
|
||||
}
|
||||
|
||||
func (store *FilerEtcStore) loadServiceAccountsFromMultiFile(ctx context.Context, s3cfg *iam_pb.S3ApiConfiguration) error {
|
||||
return store.withFilerClient(func(client filer_pb.SeaweedFilerClient) error {
|
||||
dir := filer.IamConfigDirectory + "/" + IamServiceAccountsDirectory
|
||||
entries, err := listEntries(ctx, client, dir)
|
||||
if err != nil {
|
||||
if err == filer_pb.ErrNotFound {
|
||||
return nil
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
for _, entry := range entries {
|
||||
if entry.IsDirectory {
|
||||
continue
|
||||
}
|
||||
|
||||
var content []byte
|
||||
if len(entry.Content) > 0 {
|
||||
content = entry.Content
|
||||
} else {
|
||||
c, err := filer.ReadInsideFiler(client, dir, entry.Name)
|
||||
if err != nil {
|
||||
glog.Warningf("Failed to read service account file %s: %v", entry.Name, err)
|
||||
continue
|
||||
}
|
||||
content = c
|
||||
}
|
||||
|
||||
if len(content) > 0 {
|
||||
sa := &iam_pb.ServiceAccount{}
|
||||
if err := json.Unmarshal(content, sa); err != nil {
|
||||
glog.Warningf("Failed to unmarshal service account %s: %v", entry.Name, err)
|
||||
continue
|
||||
}
|
||||
s3cfg.ServiceAccounts = append(s3cfg.ServiceAccounts, sa)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
})
|
||||
}
|
||||
|
||||
func (store *FilerEtcStore) saveServiceAccount(ctx context.Context, sa *iam_pb.ServiceAccount) error {
|
||||
if sa == nil {
|
||||
return fmt.Errorf("service account is nil")
|
||||
}
|
||||
if err := validateServiceAccountId(sa.Id); err != nil {
|
||||
return err
|
||||
}
|
||||
return store.withFilerClient(func(client filer_pb.SeaweedFilerClient) error {
|
||||
data, err := json.Marshal(sa)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return filer.SaveInsideFiler(client, filer.IamConfigDirectory+"/"+IamServiceAccountsDirectory, sa.Id+".json", data)
|
||||
})
|
||||
}
|
||||
|
||||
func (store *FilerEtcStore) deleteServiceAccount(ctx context.Context, saId string) error {
|
||||
if err := validateServiceAccountId(saId); err != nil {
|
||||
return err
|
||||
}
|
||||
return store.withFilerClient(func(client filer_pb.SeaweedFilerClient) error {
|
||||
resp, err := client.DeleteEntry(ctx, &filer_pb.DeleteEntryRequest{
|
||||
Directory: filer.IamConfigDirectory + "/" + IamServiceAccountsDirectory,
|
||||
Name: saId + ".json",
|
||||
})
|
||||
if err != nil {
|
||||
if strings.Contains(err.Error(), filer_pb.ErrNotFound.Error()) {
|
||||
return credential.ErrServiceAccountNotFound
|
||||
}
|
||||
return err
|
||||
}
|
||||
if resp != nil && resp.Error != "" {
|
||||
if strings.Contains(resp.Error, filer_pb.ErrNotFound.Error()) {
|
||||
return credential.ErrServiceAccountNotFound
|
||||
}
|
||||
return fmt.Errorf("delete service account %s: %s", saId, resp.Error)
|
||||
}
|
||||
return nil
|
||||
})
|
||||
}
|
||||
|
||||
func (store *FilerEtcStore) CreateServiceAccount(ctx context.Context, sa *iam_pb.ServiceAccount) error {
|
||||
existing, err := store.GetServiceAccount(ctx, sa.Id)
|
||||
if err != nil {
|
||||
if !errors.Is(err, credential.ErrServiceAccountNotFound) {
|
||||
return err
|
||||
}
|
||||
} else if existing != nil {
|
||||
return fmt.Errorf("service account %s already exists", sa.Id)
|
||||
}
|
||||
return store.saveServiceAccount(ctx, sa)
|
||||
}
|
||||
|
||||
func (store *FilerEtcStore) UpdateServiceAccount(ctx context.Context, id string, sa *iam_pb.ServiceAccount) error {
|
||||
if sa.Id != id {
|
||||
return fmt.Errorf("service account ID mismatch")
|
||||
}
|
||||
_, err := store.GetServiceAccount(ctx, id)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return store.saveServiceAccount(ctx, sa)
|
||||
}
|
||||
|
||||
func (store *FilerEtcStore) DeleteServiceAccount(ctx context.Context, id string) error {
|
||||
return store.deleteServiceAccount(ctx, id)
|
||||
}
|
||||
|
||||
func (store *FilerEtcStore) GetServiceAccount(ctx context.Context, id string) (*iam_pb.ServiceAccount, error) {
|
||||
if err := validateServiceAccountId(id); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var sa *iam_pb.ServiceAccount
|
||||
err := store.withFilerClient(func(client filer_pb.SeaweedFilerClient) error {
|
||||
data, err := filer.ReadInsideFiler(client, filer.IamConfigDirectory+"/"+IamServiceAccountsDirectory, id+".json")
|
||||
if err != nil {
|
||||
if err == filer_pb.ErrNotFound {
|
||||
return credential.ErrServiceAccountNotFound
|
||||
}
|
||||
return err
|
||||
}
|
||||
if len(data) == 0 {
|
||||
return credential.ErrServiceAccountNotFound
|
||||
}
|
||||
sa = &iam_pb.ServiceAccount{}
|
||||
return json.Unmarshal(data, sa)
|
||||
})
|
||||
return sa, err
|
||||
}
|
||||
|
||||
func (store *FilerEtcStore) ListServiceAccounts(ctx context.Context) ([]*iam_pb.ServiceAccount, error) {
|
||||
var accounts []*iam_pb.ServiceAccount
|
||||
err := store.withFilerClient(func(client filer_pb.SeaweedFilerClient) error {
|
||||
dir := filer.IamConfigDirectory + "/" + IamServiceAccountsDirectory
|
||||
entries, err := listEntries(ctx, client, dir)
|
||||
if err != nil {
|
||||
if err == filer_pb.ErrNotFound {
|
||||
return nil
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
for _, entry := range entries {
|
||||
if entry.IsDirectory {
|
||||
continue
|
||||
}
|
||||
|
||||
var content []byte
|
||||
if len(entry.Content) > 0 {
|
||||
content = entry.Content
|
||||
} else {
|
||||
c, err := filer.ReadInsideFiler(client, dir, entry.Name)
|
||||
if err != nil {
|
||||
glog.Warningf("Failed to read service account file %s: %v", entry.Name, err)
|
||||
continue
|
||||
}
|
||||
content = c
|
||||
}
|
||||
|
||||
if len(content) > 0 {
|
||||
sa := &iam_pb.ServiceAccount{}
|
||||
if err := json.Unmarshal(content, sa); err != nil {
|
||||
glog.Warningf("Failed to unmarshal service account %s: %v", entry.Name, err)
|
||||
continue
|
||||
}
|
||||
accounts = append(accounts, sa)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
})
|
||||
return accounts, err
|
||||
}
|
||||
|
||||
func (store *FilerEtcStore) GetServiceAccountByAccessKey(ctx context.Context, accessKey string) (*iam_pb.ServiceAccount, error) {
|
||||
accounts, err := store.ListServiceAccounts(ctx)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
for _, sa := range accounts {
|
||||
if sa.Credential != nil && sa.Credential.AccessKey == accessKey {
|
||||
return sa, nil
|
||||
}
|
||||
}
|
||||
return nil, credential.ErrAccessKeyNotFound
|
||||
}
|
||||
@@ -1,498 +0,0 @@
|
||||
package filer_multiple
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"strings"
|
||||
"sync"
|
||||
|
||||
"github.com/seaweedfs/seaweedfs/weed/credential"
|
||||
"github.com/seaweedfs/seaweedfs/weed/filer"
|
||||
"github.com/seaweedfs/seaweedfs/weed/glog"
|
||||
"github.com/seaweedfs/seaweedfs/weed/pb"
|
||||
"github.com/seaweedfs/seaweedfs/weed/pb/filer_pb"
|
||||
"github.com/seaweedfs/seaweedfs/weed/pb/iam_pb"
|
||||
"github.com/seaweedfs/seaweedfs/weed/s3api/policy_engine"
|
||||
"github.com/seaweedfs/seaweedfs/weed/util"
|
||||
"google.golang.org/grpc"
|
||||
)
|
||||
|
||||
const (
|
||||
IdentitiesDirectory = "/etc/seaweedfs/identities"
|
||||
PoliciesDirectory = "/etc/seaweedfs/policies"
|
||||
)
|
||||
|
||||
func init() {
|
||||
credential.Stores = append(credential.Stores, &FilerMultipleStore{})
|
||||
}
|
||||
|
||||
// FilerMultipleStore implements CredentialStore using SeaweedFS filer for storage
|
||||
// storing each identity in a separate file
|
||||
type FilerMultipleStore struct {
|
||||
filerAddressFunc func() pb.ServerAddress // Function to get current active filer
|
||||
grpcDialOption grpc.DialOption
|
||||
mu sync.RWMutex // Protects filerAddressFunc and grpcDialOption
|
||||
}
|
||||
|
||||
func (store *FilerMultipleStore) GetName() credential.CredentialStoreTypeName {
|
||||
return credential.StoreTypeFilerMultiple
|
||||
}
|
||||
|
||||
func (store *FilerMultipleStore) Initialize(configuration util.Configuration, prefix string) error {
|
||||
// Handle nil configuration gracefully
|
||||
if configuration != nil {
|
||||
filerAddr := configuration.GetString(prefix + "filer")
|
||||
if filerAddr != "" {
|
||||
// Static configuration - use fixed address
|
||||
store.mu.Lock()
|
||||
store.filerAddressFunc = func() pb.ServerAddress {
|
||||
return pb.ServerAddress(filerAddr)
|
||||
}
|
||||
store.mu.Unlock()
|
||||
}
|
||||
}
|
||||
// Note: filerAddressFunc can be set later via SetFilerAddressFunc method
|
||||
return nil
|
||||
}
|
||||
|
||||
// SetFilerAddressFunc sets a function that returns the current active filer address
|
||||
// This enables high availability by using the currently active filer
|
||||
func (store *FilerMultipleStore) SetFilerAddressFunc(getFiler func() pb.ServerAddress, grpcDialOption grpc.DialOption) {
|
||||
store.mu.Lock()
|
||||
defer store.mu.Unlock()
|
||||
store.filerAddressFunc = getFiler
|
||||
store.grpcDialOption = grpcDialOption
|
||||
}
|
||||
|
||||
// withFilerClient executes a function with a filer client
|
||||
func (store *FilerMultipleStore) withFilerClient(fn func(client filer_pb.SeaweedFilerClient) error) error {
|
||||
store.mu.RLock()
|
||||
if store.filerAddressFunc == nil {
|
||||
store.mu.RUnlock()
|
||||
return fmt.Errorf("filer_multiple: filer not yet available - please wait for filer discovery to complete and try again")
|
||||
}
|
||||
|
||||
filerAddress := store.filerAddressFunc()
|
||||
dialOption := store.grpcDialOption
|
||||
store.mu.RUnlock()
|
||||
|
||||
if filerAddress == "" {
|
||||
return fmt.Errorf("filer_multiple: no filer discovered yet - please ensure a filer is running and accessible")
|
||||
}
|
||||
|
||||
// Use the pb.WithGrpcFilerClient helper similar to existing code
|
||||
return pb.WithGrpcFilerClient(false, 0, filerAddress, dialOption, fn)
|
||||
}
|
||||
|
||||
func (store *FilerMultipleStore) Shutdown() {
|
||||
// No cleanup needed for file store
|
||||
}
|
||||
|
||||
func (store *FilerMultipleStore) LoadConfiguration(ctx context.Context) (*iam_pb.S3ApiConfiguration, error) {
|
||||
s3cfg := &iam_pb.S3ApiConfiguration{}
|
||||
|
||||
err := store.withFilerClient(func(client filer_pb.SeaweedFilerClient) error {
|
||||
// List and process all identity files in the directory using streaming callback
|
||||
return filer_pb.SeaweedList(ctx, client, IdentitiesDirectory, "", func(entry *filer_pb.Entry, isLast bool) error {
|
||||
if entry.IsDirectory || !strings.HasSuffix(entry.Name, ".json") {
|
||||
return nil
|
||||
}
|
||||
|
||||
content, err := filer.ReadInsideFiler(client, IdentitiesDirectory, entry.Name)
|
||||
if err != nil {
|
||||
glog.Warningf("Failed to read identity file %s: %v", entry.Name, err)
|
||||
return nil // Continue with next file
|
||||
}
|
||||
|
||||
identity := &iam_pb.Identity{}
|
||||
if err := json.Unmarshal(content, identity); err != nil {
|
||||
glog.Warningf("Failed to parse identity file %s: %v", entry.Name, err)
|
||||
return nil // Continue with next file
|
||||
}
|
||||
|
||||
s3cfg.Identities = append(s3cfg.Identities, identity)
|
||||
return nil
|
||||
}, "", false, 10000)
|
||||
})
|
||||
|
||||
if err != nil {
|
||||
// If listing failed because directory doesn't exist, treat as empty config
|
||||
if err == filer_pb.ErrNotFound {
|
||||
return s3cfg, nil
|
||||
}
|
||||
return s3cfg, err
|
||||
}
|
||||
|
||||
return s3cfg, nil
|
||||
}
|
||||
|
||||
func (store *FilerMultipleStore) SaveConfiguration(ctx context.Context, config *iam_pb.S3ApiConfiguration) error {
|
||||
// This operation is expensive for multiple files mode as it would overwrite everything
|
||||
// But we implement it for interface compliance.
|
||||
// We will write each identity to a separate file and remove stale files.
|
||||
|
||||
return store.withFilerClient(func(client filer_pb.SeaweedFilerClient) error {
|
||||
// 1. List existing identity files
|
||||
existingFileNames := make(map[string]bool)
|
||||
err := filer_pb.SeaweedList(ctx, client, IdentitiesDirectory, "", func(entry *filer_pb.Entry, isLast bool) error {
|
||||
if !entry.IsDirectory && strings.HasSuffix(entry.Name, ".json") {
|
||||
existingFileNames[entry.Name] = true
|
||||
}
|
||||
return nil
|
||||
}, "", false, 10000)
|
||||
|
||||
if err != nil && err != filer_pb.ErrNotFound {
|
||||
return fmt.Errorf("failed to list existing identities: %w", err)
|
||||
}
|
||||
|
||||
// 2. Build a set of identity keys present in the provided config
|
||||
newKeys := make(map[string]bool)
|
||||
for _, identity := range config.Identities {
|
||||
newKeys[identity.Name+".json"] = true
|
||||
}
|
||||
|
||||
// 3. Write/overwrite each identity using saveIdentity
|
||||
for _, identity := range config.Identities {
|
||||
if err := store.saveIdentity(ctx, client, identity); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
|
||||
// 4. Delete any existing files whose identity key is not in the new set
|
||||
for filename := range existingFileNames {
|
||||
if !newKeys[filename] {
|
||||
err := filer_pb.DoRemove(ctx, client, IdentitiesDirectory, filename, false, false, false, false, nil)
|
||||
if err != nil && err != filer_pb.ErrNotFound {
|
||||
glog.Warningf("failed to remove stale identity file %s: %v", filename, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return nil
|
||||
})
|
||||
}
|
||||
|
||||
func (store *FilerMultipleStore) saveIdentity(ctx context.Context, client filer_pb.SeaweedFilerClient, identity *iam_pb.Identity) error {
|
||||
data, err := json.Marshal(identity)
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to marshal identity %s: %w", identity.Name, err)
|
||||
}
|
||||
|
||||
filename := identity.Name + ".json"
|
||||
return filer.SaveInsideFiler(client, IdentitiesDirectory, filename, data)
|
||||
}
|
||||
|
||||
func (store *FilerMultipleStore) CreateUser(ctx context.Context, identity *iam_pb.Identity) error {
|
||||
return store.withFilerClient(func(client filer_pb.SeaweedFilerClient) error {
|
||||
filename := identity.Name + ".json"
|
||||
// Check if exists
|
||||
exists, err := store.exists(ctx, client, IdentitiesDirectory, filename)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if exists {
|
||||
return credential.ErrUserAlreadyExists
|
||||
}
|
||||
|
||||
return store.saveIdentity(ctx, client, identity)
|
||||
})
|
||||
}
|
||||
|
||||
func (store *FilerMultipleStore) exists(ctx context.Context, client filer_pb.SeaweedFilerClient, dir, name string) (bool, error) {
|
||||
request := &filer_pb.LookupDirectoryEntryRequest{
|
||||
Directory: dir,
|
||||
Name: name,
|
||||
}
|
||||
resp, err := filer_pb.LookupEntry(ctx, client, request)
|
||||
if err != nil {
|
||||
if err == filer_pb.ErrNotFound {
|
||||
return false, nil
|
||||
}
|
||||
return false, err
|
||||
}
|
||||
return resp.Entry != nil, nil
|
||||
}
|
||||
|
||||
func (store *FilerMultipleStore) GetUser(ctx context.Context, username string) (*iam_pb.Identity, error) {
|
||||
var identity *iam_pb.Identity
|
||||
err := store.withFilerClient(func(client filer_pb.SeaweedFilerClient) error {
|
||||
filename := username + ".json"
|
||||
content, err := filer.ReadInsideFiler(client, IdentitiesDirectory, filename)
|
||||
if err != nil {
|
||||
if err == filer_pb.ErrNotFound {
|
||||
return credential.ErrUserNotFound
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
identity = &iam_pb.Identity{}
|
||||
if err := json.Unmarshal(content, identity); err != nil {
|
||||
return fmt.Errorf("failed to parse identity: %w", err)
|
||||
}
|
||||
return nil
|
||||
})
|
||||
return identity, err
|
||||
}
|
||||
|
||||
func (store *FilerMultipleStore) UpdateUser(ctx context.Context, username string, identity *iam_pb.Identity) error {
|
||||
return store.withFilerClient(func(client filer_pb.SeaweedFilerClient) error {
|
||||
filename := username + ".json"
|
||||
// Check if exists
|
||||
exists, err := store.exists(ctx, client, IdentitiesDirectory, filename)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if !exists {
|
||||
return credential.ErrUserNotFound
|
||||
}
|
||||
|
||||
// If username changed (renamed), we need to create new file and then delete old one
|
||||
if identity.Name != username {
|
||||
// Check if the new username already exists to prevent overwrites
|
||||
newFilename := identity.Name + ".json"
|
||||
exists, err := store.exists(ctx, client, IdentitiesDirectory, newFilename)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if exists {
|
||||
return fmt.Errorf("user %s already exists", identity.Name)
|
||||
}
|
||||
|
||||
// Create new identity file FIRST
|
||||
if err := store.saveIdentity(ctx, client, identity); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
// Delete old user file SECOND
|
||||
err = filer_pb.DoRemove(ctx, client, IdentitiesDirectory, filename, false, false, false, false, nil)
|
||||
if err != nil && err != filer_pb.ErrNotFound {
|
||||
// Rollback: try to remove the newly created file if deleting the old one failed
|
||||
if errRollback := filer_pb.DoRemove(ctx, client, IdentitiesDirectory, newFilename, false, false, false, false, nil); errRollback != nil {
|
||||
glog.Errorf("Rollback of creating %s failed after failing to remove %s: %v", newFilename, filename, errRollback)
|
||||
}
|
||||
return fmt.Errorf("failed to remove old identity file %s: %w", filename, err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
return store.saveIdentity(ctx, client, identity)
|
||||
})
|
||||
}
|
||||
|
||||
func (store *FilerMultipleStore) DeleteUser(ctx context.Context, username string) error {
|
||||
return store.withFilerClient(func(client filer_pb.SeaweedFilerClient) error {
|
||||
filename := username + ".json"
|
||||
err := filer_pb.DoRemove(ctx, client, IdentitiesDirectory, filename, false, false, false, false, nil)
|
||||
if err != nil {
|
||||
if err == filer_pb.ErrNotFound {
|
||||
return nil
|
||||
}
|
||||
return err
|
||||
}
|
||||
return nil
|
||||
})
|
||||
}
|
||||
|
||||
func (store *FilerMultipleStore) ListUsers(ctx context.Context) ([]string, error) {
|
||||
var usernames []string
|
||||
err := store.withFilerClient(func(client filer_pb.SeaweedFilerClient) error {
|
||||
err := filer_pb.SeaweedList(ctx, client, IdentitiesDirectory, "", func(entry *filer_pb.Entry, isLast bool) error {
|
||||
if !entry.IsDirectory && strings.HasSuffix(entry.Name, ".json") {
|
||||
name := strings.TrimSuffix(entry.Name, ".json")
|
||||
usernames = append(usernames, name)
|
||||
}
|
||||
return nil
|
||||
}, "", false, 10000)
|
||||
|
||||
if err != nil {
|
||||
if err == filer_pb.ErrNotFound {
|
||||
// Treat as empty if directory not found
|
||||
return nil
|
||||
}
|
||||
return err
|
||||
}
|
||||
return nil
|
||||
})
|
||||
return usernames, err
|
||||
}
|
||||
|
||||
func (store *FilerMultipleStore) GetUserByAccessKey(ctx context.Context, accessKey string) (*iam_pb.Identity, error) {
|
||||
// This is inefficient in file store without index.
|
||||
// We must iterate all users.
|
||||
config, err := store.LoadConfiguration(ctx)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
for _, identity := range config.Identities {
|
||||
for _, credential := range identity.Credentials {
|
||||
if credential.AccessKey == accessKey {
|
||||
return identity, nil
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return nil, credential.ErrAccessKeyNotFound
|
||||
}
|
||||
|
||||
func (store *FilerMultipleStore) CreateAccessKey(ctx context.Context, username string, cred *iam_pb.Credential) error {
|
||||
identity, err := store.GetUser(ctx, username)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
// Check duplicates
|
||||
for _, existing := range identity.Credentials {
|
||||
if existing.AccessKey == cred.AccessKey {
|
||||
return fmt.Errorf("access key already exists")
|
||||
}
|
||||
}
|
||||
|
||||
identity.Credentials = append(identity.Credentials, cred)
|
||||
return store.UpdateUser(ctx, username, identity)
|
||||
}
|
||||
|
||||
func (store *FilerMultipleStore) DeleteAccessKey(ctx context.Context, username string, accessKey string) error {
|
||||
identity, err := store.GetUser(ctx, username)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
found := false
|
||||
for i, cred := range identity.Credentials {
|
||||
if cred.AccessKey == accessKey {
|
||||
identity.Credentials = append(identity.Credentials[:i], identity.Credentials[i+1:]...)
|
||||
found = true
|
||||
break
|
||||
}
|
||||
}
|
||||
|
||||
if !found {
|
||||
return credential.ErrAccessKeyNotFound
|
||||
}
|
||||
|
||||
return store.UpdateUser(ctx, username, identity)
|
||||
}
|
||||
|
||||
// PolicyManager implementation
|
||||
|
||||
func (store *FilerMultipleStore) GetPolicies(ctx context.Context) (map[string]policy_engine.PolicyDocument, error) {
|
||||
policies := make(map[string]policy_engine.PolicyDocument)
|
||||
|
||||
err := store.withFilerClient(func(client filer_pb.SeaweedFilerClient) error {
|
||||
return filer_pb.SeaweedList(ctx, client, PoliciesDirectory, "", func(entry *filer_pb.Entry, isLast bool) error {
|
||||
if entry.IsDirectory || !strings.HasSuffix(entry.Name, ".json") {
|
||||
return nil
|
||||
}
|
||||
|
||||
content, err := filer.ReadInsideFiler(client, PoliciesDirectory, entry.Name)
|
||||
if err != nil {
|
||||
glog.Warningf("Failed to read policy file %s: %v", entry.Name, err)
|
||||
return nil
|
||||
}
|
||||
|
||||
var policy policy_engine.PolicyDocument
|
||||
if err := json.Unmarshal(content, &policy); err != nil {
|
||||
glog.Warningf("Failed to parse policy file %s: %v", entry.Name, err)
|
||||
return nil
|
||||
}
|
||||
|
||||
name := strings.TrimSuffix(entry.Name, ".json")
|
||||
policies[name] = policy
|
||||
return nil
|
||||
}, "", false, 10000)
|
||||
})
|
||||
|
||||
if err != nil {
|
||||
if err == filer_pb.ErrNotFound {
|
||||
return policies, nil
|
||||
}
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return policies, nil
|
||||
}
|
||||
|
||||
func (store *FilerMultipleStore) CreatePolicy(ctx context.Context, name string, document policy_engine.PolicyDocument) error {
|
||||
return store.withFilerClient(func(client filer_pb.SeaweedFilerClient) error {
|
||||
filename := name + ".json"
|
||||
exists, err := store.exists(ctx, client, PoliciesDirectory, filename)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if exists {
|
||||
return fmt.Errorf("policy %s already exists", name)
|
||||
}
|
||||
|
||||
return store.savePolicy(ctx, client, name, document)
|
||||
})
|
||||
}
|
||||
|
||||
func (store *FilerMultipleStore) PutPolicy(ctx context.Context, name string, document policy_engine.PolicyDocument) error {
|
||||
return store.withFilerClient(func(client filer_pb.SeaweedFilerClient) error {
|
||||
// We can just overwrite. The distinction between Create and Update in filer_multiple was just checking existence.
|
||||
// Put implies "create or replace".
|
||||
return store.savePolicy(ctx, client, name, document)
|
||||
})
|
||||
}
|
||||
|
||||
func (store *FilerMultipleStore) UpdatePolicy(ctx context.Context, name string, document policy_engine.PolicyDocument) error {
|
||||
return store.withFilerClient(func(client filer_pb.SeaweedFilerClient) error {
|
||||
filename := name + ".json"
|
||||
exists, err := store.exists(ctx, client, PoliciesDirectory, filename)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if !exists {
|
||||
return fmt.Errorf("policy %s not found", name)
|
||||
}
|
||||
|
||||
return store.savePolicy(ctx, client, name, document)
|
||||
})
|
||||
}
|
||||
|
||||
func (store *FilerMultipleStore) DeletePolicy(ctx context.Context, name string) error {
|
||||
return store.withFilerClient(func(client filer_pb.SeaweedFilerClient) error {
|
||||
filename := name + ".json"
|
||||
err := filer_pb.DoRemove(ctx, client, PoliciesDirectory, filename, false, false, false, false, nil)
|
||||
if err != nil {
|
||||
if err == filer_pb.ErrNotFound {
|
||||
return nil
|
||||
}
|
||||
return err
|
||||
}
|
||||
return nil
|
||||
})
|
||||
}
|
||||
|
||||
func (store *FilerMultipleStore) GetPolicy(ctx context.Context, name string) (*policy_engine.PolicyDocument, error) {
|
||||
var policy *policy_engine.PolicyDocument
|
||||
err := store.withFilerClient(func(client filer_pb.SeaweedFilerClient) error {
|
||||
filename := name + ".json"
|
||||
content, err := filer.ReadInsideFiler(client, PoliciesDirectory, filename)
|
||||
if err != nil {
|
||||
if err == filer_pb.ErrNotFound {
|
||||
return nil
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
policy = &policy_engine.PolicyDocument{}
|
||||
if err := json.Unmarshal(content, policy); err != nil {
|
||||
return fmt.Errorf("failed to parse policy: %w", err)
|
||||
}
|
||||
return nil
|
||||
})
|
||||
return policy, err
|
||||
}
|
||||
|
||||
func (store *FilerMultipleStore) savePolicy(ctx context.Context, client filer_pb.SeaweedFilerClient, name string, document policy_engine.PolicyDocument) error {
|
||||
data, err := json.Marshal(document)
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to marshal policy %s: %w", name, err)
|
||||
}
|
||||
|
||||
filename := name + ".json"
|
||||
return filer.SaveInsideFiler(client, PoliciesDirectory, filename, data)
|
||||
}
|
||||
@@ -7,6 +7,8 @@ import (
|
||||
|
||||
"github.com/seaweedfs/seaweedfs/weed/pb/iam_pb"
|
||||
"github.com/seaweedfs/seaweedfs/weed/s3api/policy_engine"
|
||||
"google.golang.org/grpc/codes"
|
||||
"google.golang.org/grpc/status"
|
||||
)
|
||||
|
||||
func (store *IamGrpcStore) GetPolicies(ctx context.Context) (map[string]policy_engine.PolicyDocument, error) {
|
||||
@@ -63,7 +65,21 @@ func (store *IamGrpcStore) GetPolicy(ctx context.Context, name string) (*policy_
|
||||
return json.Unmarshal([]byte(resp.Content), &doc)
|
||||
})
|
||||
if err != nil {
|
||||
// If policy not found, return nil instead of error (consistent with other stores)
|
||||
if st, ok := status.FromError(err); ok && st.Code() == codes.NotFound {
|
||||
return nil, nil
|
||||
}
|
||||
return nil, err
|
||||
}
|
||||
return &doc, nil
|
||||
}
|
||||
|
||||
// CreatePolicy creates a new policy (delegates to PutPolicy)
|
||||
func (store *IamGrpcStore) CreatePolicy(ctx context.Context, name string, document policy_engine.PolicyDocument) error {
|
||||
return store.PutPolicy(ctx, name, document)
|
||||
}
|
||||
|
||||
// UpdatePolicy updates an existing policy (delegates to PutPolicy)
|
||||
func (store *IamGrpcStore) UpdatePolicy(ctx context.Context, name string, document policy_engine.PolicyDocument) error {
|
||||
return store.PutPolicy(ctx, name, document)
|
||||
}
|
||||
|
||||
78
weed/credential/grpc/grpc_service_account.go
Normal file
78
weed/credential/grpc/grpc_service_account.go
Normal file
@@ -0,0 +1,78 @@
|
||||
package grpc
|
||||
|
||||
import (
|
||||
"context"
|
||||
|
||||
"github.com/seaweedfs/seaweedfs/weed/pb/iam_pb"
|
||||
)
|
||||
|
||||
func (store *IamGrpcStore) CreateServiceAccount(ctx context.Context, sa *iam_pb.ServiceAccount) error {
|
||||
return store.withIamClient(func(client iam_pb.SeaweedIdentityAccessManagementClient) error {
|
||||
_, err := client.CreateServiceAccount(ctx, &iam_pb.CreateServiceAccountRequest{
|
||||
ServiceAccount: sa,
|
||||
})
|
||||
return err
|
||||
})
|
||||
}
|
||||
|
||||
func (store *IamGrpcStore) UpdateServiceAccount(ctx context.Context, id string, sa *iam_pb.ServiceAccount) error {
|
||||
return store.withIamClient(func(client iam_pb.SeaweedIdentityAccessManagementClient) error {
|
||||
_, err := client.UpdateServiceAccount(ctx, &iam_pb.UpdateServiceAccountRequest{
|
||||
Id: id,
|
||||
ServiceAccount: sa,
|
||||
})
|
||||
return err
|
||||
})
|
||||
}
|
||||
|
||||
func (store *IamGrpcStore) DeleteServiceAccount(ctx context.Context, id string) error {
|
||||
return store.withIamClient(func(client iam_pb.SeaweedIdentityAccessManagementClient) error {
|
||||
_, err := client.DeleteServiceAccount(ctx, &iam_pb.DeleteServiceAccountRequest{
|
||||
Id: id,
|
||||
})
|
||||
return err
|
||||
})
|
||||
}
|
||||
|
||||
func (store *IamGrpcStore) GetServiceAccount(ctx context.Context, id string) (*iam_pb.ServiceAccount, error) {
|
||||
var sa *iam_pb.ServiceAccount
|
||||
err := store.withIamClient(func(client iam_pb.SeaweedIdentityAccessManagementClient) error {
|
||||
resp, err := client.GetServiceAccount(ctx, &iam_pb.GetServiceAccountRequest{
|
||||
Id: id,
|
||||
})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
sa = resp.ServiceAccount
|
||||
return nil
|
||||
})
|
||||
return sa, err
|
||||
}
|
||||
|
||||
func (store *IamGrpcStore) ListServiceAccounts(ctx context.Context) ([]*iam_pb.ServiceAccount, error) {
|
||||
var accounts []*iam_pb.ServiceAccount
|
||||
err := store.withIamClient(func(client iam_pb.SeaweedIdentityAccessManagementClient) error {
|
||||
resp, err := client.ListServiceAccounts(ctx, &iam_pb.ListServiceAccountsRequest{})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
accounts = resp.ServiceAccounts
|
||||
return nil
|
||||
})
|
||||
return accounts, err
|
||||
}
|
||||
|
||||
func (store *IamGrpcStore) GetServiceAccountByAccessKey(ctx context.Context, accessKey string) (*iam_pb.ServiceAccount, error) {
|
||||
var sa *iam_pb.ServiceAccount
|
||||
err := store.withIamClient(func(client iam_pb.SeaweedIdentityAccessManagementClient) error {
|
||||
resp, err := client.GetServiceAccountByAccessKey(ctx, &iam_pb.GetServiceAccountByAccessKeyRequest{
|
||||
AccessKey: accessKey,
|
||||
})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
sa = resp.ServiceAccount
|
||||
return nil
|
||||
})
|
||||
return sa, err
|
||||
}
|
||||
85
weed/credential/memory/memory_service_account.go
Normal file
85
weed/credential/memory/memory_service_account.go
Normal file
@@ -0,0 +1,85 @@
|
||||
package memory
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
|
||||
"github.com/seaweedfs/seaweedfs/weed/credential"
|
||||
"github.com/seaweedfs/seaweedfs/weed/pb/iam_pb"
|
||||
)
|
||||
|
||||
func (store *MemoryStore) CreateServiceAccount(ctx context.Context, sa *iam_pb.ServiceAccount) error {
|
||||
store.mu.Lock()
|
||||
defer store.mu.Unlock()
|
||||
|
||||
if _, exists := store.serviceAccounts[sa.Id]; exists {
|
||||
return fmt.Errorf("service account already exists")
|
||||
}
|
||||
store.serviceAccounts[sa.Id] = sa
|
||||
if sa.Credential != nil && sa.Credential.AccessKey != "" {
|
||||
store.serviceAccountAccessKeys[sa.Credential.AccessKey] = sa.Id
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (store *MemoryStore) UpdateServiceAccount(ctx context.Context, id string, sa *iam_pb.ServiceAccount) error {
|
||||
store.mu.Lock()
|
||||
defer store.mu.Unlock()
|
||||
|
||||
_, exists := store.serviceAccounts[id]
|
||||
if !exists {
|
||||
return credential.ErrServiceAccountNotFound
|
||||
}
|
||||
if sa.Id != id {
|
||||
return fmt.Errorf("service account ID mismatch")
|
||||
}
|
||||
|
||||
// Update access key index: remove any existing keys for this SA
|
||||
for k, v := range store.serviceAccountAccessKeys {
|
||||
if v == id {
|
||||
delete(store.serviceAccountAccessKeys, k)
|
||||
}
|
||||
}
|
||||
|
||||
store.serviceAccounts[id] = sa
|
||||
|
||||
if sa.Credential != nil && sa.Credential.AccessKey != "" {
|
||||
store.serviceAccountAccessKeys[sa.Credential.AccessKey] = sa.Id
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (store *MemoryStore) DeleteServiceAccount(ctx context.Context, id string) error {
|
||||
store.mu.Lock()
|
||||
defer store.mu.Unlock()
|
||||
|
||||
if sa, ok := store.serviceAccounts[id]; ok {
|
||||
if sa.Credential != nil && sa.Credential.AccessKey != "" {
|
||||
delete(store.serviceAccountAccessKeys, sa.Credential.AccessKey)
|
||||
}
|
||||
delete(store.serviceAccounts, id)
|
||||
return nil
|
||||
}
|
||||
return credential.ErrServiceAccountNotFound
|
||||
}
|
||||
|
||||
func (store *MemoryStore) GetServiceAccount(ctx context.Context, id string) (*iam_pb.ServiceAccount, error) {
|
||||
store.mu.RLock()
|
||||
defer store.mu.RUnlock()
|
||||
|
||||
if sa, exists := store.serviceAccounts[id]; exists {
|
||||
return sa, nil
|
||||
}
|
||||
return nil, credential.ErrServiceAccountNotFound
|
||||
}
|
||||
|
||||
func (store *MemoryStore) ListServiceAccounts(ctx context.Context) ([]*iam_pb.ServiceAccount, error) {
|
||||
store.mu.RLock()
|
||||
defer store.mu.RUnlock()
|
||||
|
||||
var accounts []*iam_pb.ServiceAccount
|
||||
for _, sa := range store.serviceAccounts {
|
||||
accounts = append(accounts, sa)
|
||||
}
|
||||
return accounts, nil
|
||||
}
|
||||
@@ -1,6 +1,7 @@
|
||||
package memory
|
||||
|
||||
import (
|
||||
"context"
|
||||
"sync"
|
||||
|
||||
"github.com/seaweedfs/seaweedfs/weed/credential"
|
||||
@@ -16,11 +17,13 @@ func init() {
|
||||
// MemoryStore implements CredentialStore using in-memory storage
|
||||
// This is primarily intended for testing purposes
|
||||
type MemoryStore struct {
|
||||
mu sync.RWMutex
|
||||
users map[string]*iam_pb.Identity // username -> identity
|
||||
accessKeys map[string]string // access_key -> username
|
||||
policies map[string]policy_engine.PolicyDocument // policy_name -> policy_document
|
||||
initialized bool
|
||||
mu sync.RWMutex
|
||||
users map[string]*iam_pb.Identity // username -> identity
|
||||
accessKeys map[string]string // access_key -> username
|
||||
serviceAccounts map[string]*iam_pb.ServiceAccount // id -> service_account
|
||||
serviceAccountAccessKeys map[string]string // access_key -> id
|
||||
policies map[string]policy_engine.PolicyDocument // policy_name -> policy_document
|
||||
initialized bool
|
||||
}
|
||||
|
||||
func (store *MemoryStore) GetName() credential.CredentialStoreTypeName {
|
||||
@@ -37,6 +40,8 @@ func (store *MemoryStore) Initialize(configuration util.Configuration, prefix st
|
||||
|
||||
store.users = make(map[string]*iam_pb.Identity)
|
||||
store.accessKeys = make(map[string]string)
|
||||
store.serviceAccounts = make(map[string]*iam_pb.ServiceAccount)
|
||||
store.serviceAccountAccessKeys = make(map[string]string)
|
||||
store.policies = make(map[string]policy_engine.PolicyDocument)
|
||||
store.initialized = true
|
||||
|
||||
@@ -49,6 +54,8 @@ func (store *MemoryStore) Shutdown() {
|
||||
|
||||
store.users = nil
|
||||
store.accessKeys = nil
|
||||
store.serviceAccounts = nil
|
||||
store.serviceAccountAccessKeys = nil
|
||||
store.policies = nil
|
||||
store.initialized = false
|
||||
}
|
||||
@@ -61,6 +68,9 @@ func (store *MemoryStore) Reset() {
|
||||
if store.initialized {
|
||||
store.users = make(map[string]*iam_pb.Identity)
|
||||
store.accessKeys = make(map[string]string)
|
||||
store.serviceAccounts = make(map[string]*iam_pb.ServiceAccount)
|
||||
store.serviceAccountAccessKeys = make(map[string]string)
|
||||
store.policies = make(map[string]policy_engine.PolicyDocument)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -79,3 +89,12 @@ func (store *MemoryStore) GetAccessKeyCount() int {
|
||||
|
||||
return len(store.accessKeys)
|
||||
}
|
||||
func (store *MemoryStore) GetServiceAccountByAccessKey(ctx context.Context, accessKey string) (*iam_pb.ServiceAccount, error) {
|
||||
store.mu.RLock()
|
||||
defer store.mu.RUnlock()
|
||||
|
||||
if id, ok := store.serviceAccountAccessKeys[accessKey]; ok {
|
||||
return store.serviceAccounts[id], nil
|
||||
}
|
||||
return nil, credential.ErrAccessKeyNotFound
|
||||
}
|
||||
|
||||
@@ -313,3 +313,68 @@ func TestMemoryStoreConfigurationSaveLoad(t *testing.T) {
|
||||
t.Errorf("User2 credentials not correct: %+v", user2.Credentials)
|
||||
}
|
||||
}
|
||||
|
||||
func TestMemoryStoreServiceAccountByAccessKey(t *testing.T) {
|
||||
store := &MemoryStore{}
|
||||
config := util.GetViper()
|
||||
if err := store.Initialize(config, "credential."); err != nil {
|
||||
t.Fatalf("Failed to initialize store: %v", err)
|
||||
}
|
||||
|
||||
ctx := context.Background()
|
||||
|
||||
// 1. Create service account
|
||||
sa := &iam_pb.ServiceAccount{
|
||||
Id: "sa-test-1",
|
||||
ParentUser: "user1",
|
||||
Credential: &iam_pb.Credential{
|
||||
AccessKey: "ACCESS-KEY-1",
|
||||
SecretKey: "SECRET-KEY-1",
|
||||
},
|
||||
}
|
||||
|
||||
if err := store.CreateServiceAccount(ctx, sa); err != nil {
|
||||
t.Fatalf("Failed to create service account: %v", err)
|
||||
}
|
||||
|
||||
// 2. Lookup by access key
|
||||
found, err := store.GetServiceAccountByAccessKey(ctx, "ACCESS-KEY-1")
|
||||
if err != nil {
|
||||
t.Fatalf("Failed to lookup by access key: %v", err)
|
||||
}
|
||||
if found.Id != "sa-test-1" {
|
||||
t.Errorf("Expected sa-test-1, got %s", found.Id)
|
||||
}
|
||||
|
||||
// 3. Update with new access key
|
||||
sa.Credential.AccessKey = "ACCESS-KEY-2"
|
||||
if err := store.UpdateServiceAccount(ctx, sa.Id, sa); err != nil {
|
||||
t.Fatalf("Failed to update service account: %v", err)
|
||||
}
|
||||
|
||||
// Verify old key is gone
|
||||
_, err = store.GetServiceAccountByAccessKey(ctx, "ACCESS-KEY-1")
|
||||
if err != credential.ErrAccessKeyNotFound {
|
||||
t.Errorf("Expected ErrAccessKeyNotFound for old key, got %v", err)
|
||||
}
|
||||
|
||||
// Verify new key works
|
||||
found, err = store.GetServiceAccountByAccessKey(ctx, "ACCESS-KEY-2")
|
||||
if err != nil {
|
||||
t.Fatalf("Failed to lookup by new access key: %v", err)
|
||||
}
|
||||
if found.Id != "sa-test-1" {
|
||||
t.Errorf("Expected sa-test-1, got %s", found.Id)
|
||||
}
|
||||
|
||||
// 4. Delete service account
|
||||
if err := store.DeleteServiceAccount(ctx, sa.Id); err != nil {
|
||||
t.Fatalf("Failed to delete service account: %v", err)
|
||||
}
|
||||
|
||||
// Verify key is gone
|
||||
_, err = store.GetServiceAccountByAccessKey(ctx, "ACCESS-KEY-2")
|
||||
if err != credential.ErrAccessKeyNotFound {
|
||||
t.Errorf("Expected ErrAccessKeyNotFound after delete, got %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
173
weed/credential/postgres/postgres_service_account.go
Normal file
173
weed/credential/postgres/postgres_service_account.go
Normal file
@@ -0,0 +1,173 @@
|
||||
package postgres
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
|
||||
"github.com/seaweedfs/seaweedfs/weed/credential"
|
||||
"github.com/seaweedfs/seaweedfs/weed/pb/iam_pb"
|
||||
)
|
||||
|
||||
func (store *PostgresStore) CreateServiceAccount(ctx context.Context, sa *iam_pb.ServiceAccount) error {
|
||||
if sa == nil {
|
||||
return fmt.Errorf("service account is nil")
|
||||
}
|
||||
if sa.Id == "" {
|
||||
return fmt.Errorf("service account ID is required")
|
||||
}
|
||||
if !store.configured {
|
||||
return fmt.Errorf("store not configured")
|
||||
}
|
||||
|
||||
data, err := json.Marshal(sa)
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to marshal service account: %w", err)
|
||||
}
|
||||
|
||||
accessKey := ""
|
||||
if sa.Credential != nil {
|
||||
accessKey = sa.Credential.AccessKey
|
||||
}
|
||||
|
||||
_, err = store.db.ExecContext(ctx,
|
||||
"INSERT INTO service_accounts (id, access_key, content) VALUES ($1, $2, $3)",
|
||||
sa.Id, accessKey, data)
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to insert service account: %w", err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (store *PostgresStore) UpdateServiceAccount(ctx context.Context, id string, sa *iam_pb.ServiceAccount) error {
|
||||
if sa == nil {
|
||||
return fmt.Errorf("service account is nil")
|
||||
}
|
||||
if id == "" {
|
||||
return fmt.Errorf("service account ID is required")
|
||||
}
|
||||
if sa.Id != id {
|
||||
return fmt.Errorf("service account ID mismatch")
|
||||
}
|
||||
|
||||
data, err := json.Marshal(sa)
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to marshal service account: %w", err)
|
||||
}
|
||||
|
||||
accessKey := ""
|
||||
if sa.Credential != nil {
|
||||
accessKey = sa.Credential.AccessKey
|
||||
}
|
||||
|
||||
result, err := store.db.ExecContext(ctx,
|
||||
"UPDATE service_accounts SET access_key = $2, content = $3, updated_at = CURRENT_TIMESTAMP WHERE id = $1",
|
||||
id, accessKey, data)
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to update service account: %w", err)
|
||||
}
|
||||
|
||||
rows, err := result.RowsAffected()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if rows == 0 {
|
||||
return credential.ErrServiceAccountNotFound
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (store *PostgresStore) DeleteServiceAccount(ctx context.Context, id string) error {
|
||||
if !store.configured {
|
||||
return fmt.Errorf("store not configured")
|
||||
}
|
||||
|
||||
result, err := store.db.ExecContext(ctx, "DELETE FROM service_accounts WHERE id = $1", id)
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to delete service account: %w", err)
|
||||
}
|
||||
|
||||
rows, err := result.RowsAffected()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if rows == 0 {
|
||||
return credential.ErrServiceAccountNotFound
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (store *PostgresStore) GetServiceAccount(ctx context.Context, id string) (*iam_pb.ServiceAccount, error) {
|
||||
if !store.configured {
|
||||
return nil, fmt.Errorf("store not configured")
|
||||
}
|
||||
|
||||
var content []byte
|
||||
err := store.db.QueryRowContext(ctx, "SELECT content FROM service_accounts WHERE id = $1", id).Scan(&content)
|
||||
if err != nil {
|
||||
if err == sql.ErrNoRows {
|
||||
return nil, credential.ErrServiceAccountNotFound
|
||||
}
|
||||
return nil, fmt.Errorf("failed to get service account: %w", err)
|
||||
}
|
||||
|
||||
sa := &iam_pb.ServiceAccount{}
|
||||
if err := json.Unmarshal(content, sa); err != nil {
|
||||
return nil, fmt.Errorf("failed to unmarshal service account: %w", err)
|
||||
}
|
||||
return sa, nil
|
||||
}
|
||||
|
||||
func (store *PostgresStore) ListServiceAccounts(ctx context.Context) ([]*iam_pb.ServiceAccount, error) {
|
||||
if !store.configured {
|
||||
return nil, fmt.Errorf("store not configured")
|
||||
}
|
||||
|
||||
rows, err := store.db.QueryContext(ctx, "SELECT content FROM service_accounts")
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to list service accounts: %w", err)
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
var accounts []*iam_pb.ServiceAccount
|
||||
for rows.Next() {
|
||||
var content []byte
|
||||
if err := rows.Scan(&content); err != nil {
|
||||
return nil, fmt.Errorf("failed to scan service account: %w", err)
|
||||
}
|
||||
sa := &iam_pb.ServiceAccount{}
|
||||
if err := json.Unmarshal(content, sa); err != nil {
|
||||
return nil, fmt.Errorf("failed to unmarshal service account: %w", err)
|
||||
}
|
||||
accounts = append(accounts, sa)
|
||||
}
|
||||
|
||||
if err := rows.Err(); err != nil {
|
||||
return nil, fmt.Errorf("error iterating service accounts: %w", err)
|
||||
}
|
||||
|
||||
return accounts, nil
|
||||
}
|
||||
|
||||
func (store *PostgresStore) GetServiceAccountByAccessKey(ctx context.Context, accessKey string) (*iam_pb.ServiceAccount, error) {
|
||||
if !store.configured {
|
||||
return nil, fmt.Errorf("store not configured")
|
||||
}
|
||||
|
||||
var content []byte
|
||||
err := store.db.QueryRowContext(ctx, "SELECT content FROM service_accounts WHERE access_key = $1", accessKey).Scan(&content)
|
||||
if err != nil {
|
||||
if err == sql.ErrNoRows {
|
||||
return nil, credential.ErrAccessKeyNotFound
|
||||
}
|
||||
return nil, fmt.Errorf("failed to query service account by access key: %w", err)
|
||||
}
|
||||
|
||||
sa := &iam_pb.ServiceAccount{}
|
||||
if err := json.Unmarshal(content, sa); err != nil {
|
||||
return nil, fmt.Errorf("failed to unmarshal service account: %w", err)
|
||||
}
|
||||
|
||||
return sa, nil
|
||||
}
|
||||
@@ -123,6 +123,17 @@ func (store *PostgresStore) createTables() error {
|
||||
CREATE INDEX IF NOT EXISTS idx_policies_name ON policies(name);
|
||||
`
|
||||
|
||||
// Create service_accounts table
|
||||
serviceAccountsTable := `
|
||||
CREATE TABLE IF NOT EXISTS service_accounts (
|
||||
id VARCHAR(255) PRIMARY KEY,
|
||||
access_key VARCHAR(255) UNIQUE,
|
||||
content JSONB NOT NULL,
|
||||
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
|
||||
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
|
||||
);
|
||||
`
|
||||
|
||||
// Execute table creation
|
||||
if _, err := store.db.Exec(usersTable); err != nil {
|
||||
return fmt.Errorf("failed to create users table: %w", err)
|
||||
@@ -136,6 +147,10 @@ func (store *PostgresStore) createTables() error {
|
||||
return fmt.Errorf("failed to create policies table: %w", err)
|
||||
}
|
||||
|
||||
if _, err := store.db.Exec(serviceAccountsTable); err != nil {
|
||||
return fmt.Errorf("failed to create service_accounts table: %w", err)
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
|
||||
28
weed/credential/validation.go
Normal file
28
weed/credential/validation.go
Normal file
@@ -0,0 +1,28 @@
|
||||
package credential
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"regexp"
|
||||
)
|
||||
|
||||
var (
|
||||
PolicyNamePattern = regexp.MustCompile(`^[A-Za-z0-9_-]+$`)
|
||||
ServiceAccountIdPattern = regexp.MustCompile(`^sa:[A-Za-z0-9_-]+:[a-z0-9-]+$`)
|
||||
)
|
||||
|
||||
func ValidatePolicyName(name string) error {
|
||||
if !PolicyNamePattern.MatchString(name) {
|
||||
return fmt.Errorf("invalid policy name: %s", name)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func ValidateServiceAccountId(id string) error {
|
||||
if id == "" {
|
||||
return fmt.Errorf("service account ID cannot be empty")
|
||||
}
|
||||
if !ServiceAccountIdPattern.MatchString(id) {
|
||||
return fmt.Errorf("invalid service account ID: %s (expected format sa:<user>:<uuid>)", id)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
Reference in New Issue
Block a user