Files
seaweedFS/weed/s3api/auth_credentials.go
Chris Lu ae9a943ef6 IAM: Add Service Account Support (#7744) (#7901)
* iam: add ServiceAccount protobuf schema

Add ServiceAccount message type to iam.proto with support for:
- Unique ID and parent user linkage
- Optional expiration timestamp
- Separate credentials (access key/secret)
- Action restrictions (subset of parent)
- Enable/disable status

This is the first step toward implementing issue #7744
(IAM Service Account Support).

* iam: add service account response types

Add IAM API response types for service account operations:
- ServiceAccountInfo struct for marshaling account details
- CreateServiceAccountResponse
- DeleteServiceAccountResponse
- ListServiceAccountsResponse
- GetServiceAccountResponse
- UpdateServiceAccountResponse

Also add type aliases in iamapi package for backwards compatibility.

Part of issue #7744 (IAM Service Account Support).

* iam: implement service account API handlers

Add CRUD operations for service accounts:
- CreateServiceAccount: Creates service account with ABIA key prefix
- DeleteServiceAccount: Removes service account and parent linkage
- ListServiceAccounts: Lists all or filtered by parent user
- GetServiceAccount: Retrieves service account details
- UpdateServiceAccount: Modifies status, description, expiration

Service accounts inherit parent user's actions by default and
support optional expiration timestamps.

Part of issue #7744 (IAM Service Account Support).

* sts: add AssumeRoleWithWebIdentity HTTP endpoint

Add STS API HTTP endpoint for AWS SDK compatibility:
- Create s3api_sts.go with HTTP handlers matching AWS STS spec
- Support AssumeRoleWithWebIdentity action with JWT token
- Return XML response with temporary credentials (AccessKeyId,
  SecretAccessKey, SessionToken) matching AWS format
- Register STS route at POST /?Action=AssumeRoleWithWebIdentity

This enables AWS SDKs (boto3, AWS CLI, etc.) to obtain temporary
S3 credentials using OIDC/JWT tokens.

Part of issue #7744 (IAM Service Account Support).

* test: add service account and STS integration tests

Add integration tests for new IAM features:

s3_service_account_test.go:
- TestServiceAccountLifecycle: Create, Get, List, Update, Delete
- TestServiceAccountValidation: Error handling for missing params

s3_sts_test.go:
- TestAssumeRoleWithWebIdentityValidation: Parameter validation
- TestAssumeRoleWithWebIdentityWithMockJWT: JWT token handling

Tests skip gracefully when SeaweedFS is not running or when IAM
features are not configured.

Part of issue #7744 (IAM Service Account Support).

* iam: address code review comments

- Add constants for service account ID and key lengths
- Use strconv.ParseInt instead of fmt.Sscanf for better error handling
- Allow clearing descriptions by checking key existence in url.Values
- Replace magic numbers (12, 20, 40) with named constants

Addresses review comments from gemini-code-assist[bot]

* test: add proper error handling in service account tests

Use require.NoError(t, err) for io.ReadAll and xml.Unmarshal
to prevent silent failures and ensure test reliability.

Addresses review comment from gemini-code-assist[bot]

* test: add proper error handling in STS tests

Use require.NoError(t, err) for io.ReadAll and xml.Unmarshal
to prevent silent failures and ensure test reliability.
Repeated this fix throughout the file.

Addresses review comment from gemini-code-assist[bot] in PR #7901.

* iam: address additional code review comments

- Specific error code mapping for STS service errors
- Distinguish between Sender and Receiver error types in STS responses
- Add nil checks for credentials in List/GetServiceAccount
- Validate expiration date is in the future
- Improve integration test error messages (include response body)
- Add credential verification step in service account tests

Addresses remaining review comments from gemini-code-assist[bot] across multiple files.

* iam: fix shared slice reference in service account creation

Copy parent's actions to create an independent slice for the service
account instead of sharing the underlying array. This prevents
unexpected mutations when the parent's actions are modified later.

Addresses review comment from coderabbitai[bot] in PR #7901.

* iam: remove duplicate unused constant

Removed redundant iamServiceAccountKeyPrefix as ServiceAccountKeyPrefix
is already defined and used.

Addresses remaining cleanup task.

* sts: document limitation of string-based error mapping

Added TODO comment explaining that the current string-based error
mapping approach is fragile and should be replaced with typed errors
from the STS service in a future refactoring.

This addresses the architectural concern raised in code review while
deferring the actual implementation to a separate PR to avoid scope
creep in the current service account feature addition.

* iam: fix remaining review issues

- Add future-date validation for expiration in UpdateServiceAccount
- Reorder tests so credential verification happens before deletion
- Fix compilation error by using correct JWT generation methods

Addresses final review comments from coderabbitai[bot].

* iam: fix service account access key length

The access key IDs were incorrectly generated with 24 characters
instead of the AWS-standard 20 characters. This was caused by
generating 20 random characters and then prepending the 4-character
ABIA prefix.

Fixed by subtracting the prefix length from AccessKeyLength, so the
final key is: ABIA (4 chars) + random (16 chars) = 20 chars total.

This ensures compatibility with S3 clients that validate key length.

* test: add comprehensive service account security tests

Added comprehensive integration tests for service account functionality:

- TestServiceAccountS3Access: Verify SA credentials work for S3 operations
- TestServiceAccountExpiration: Test expiration date validation and enforcement
- TestServiceAccountInheritedPermissions: Verify parent-child relationship
- TestServiceAccountAccessKeyFormat: Validate AWS-compatible key format (ABIA prefix, 20 char length)

These tests ensure SeaweedFS service accounts are compatible with AWS
conventions and provide robust security coverage.

* iam: remove unused UserAccessKeyPrefix constant

Code cleanup to remove unused constants.

* iam: remove unused iamCommonResponse type alias

Code cleanup to remove unused type aliases.

* iam: restore and use UserAccessKeyPrefix constant

Restored UserAccessKeyPrefix constant and updated s3api tests to use it
instead of hardcoded strings for better maintainability and consistency.

* test: improve error handling in service account security tests

Added explicit error checking for io.ReadAll and xml.Unmarshal in
TestServiceAccountExpiration to ensure failures are reported correctly and
cleanup is performed only when appropriate. Also added logging for failed
responses.

* test: use t.Cleanup for reliable resource cleanup

Replaced defer with t.Cleanup to ensure service account cleanup runs even
when require.NoError fails. Also switched from manual error checking to
require.NoError for more idiomatic testify usage.

* iam: add CreatedBy field and optimize identity lookups

- Added createdBy parameter to CreateServiceAccount to track who created each service account
- Extract creator identity from request context using GetIdentityNameFromContext
- Populate created_by field in ServiceAccount protobuf
- Added findIdentityByName helper function to optimize identity lookups
- Replaced nested loops with O(n) helper function calls in CreateServiceAccount and DeleteServiceAccount

This addresses code review feedback for better auditing and performance.

* iam: prevent user deletion when service accounts exist

Following AWS IAM behavior, prevent deletion of users that have active
service accounts. This ensures explicit cleanup and prevents orphaned
service account resources with invalid ParentUser references.

Users must delete all associated service accounts before deleting the
parent user, providing safer resource management.

* sts: enhance TODO with typed error implementation guidance

Updated TODO comment with detailed implementation approach for replacing
string-based error matching with typed errors using errors.Is(). This
provides a clear roadmap for a follow-up PR to improve error handling
robustness and maintainability.

* iam: add operational limits for service account creation

Added AWS IAM-compatible safeguards to prevent resource exhaustion:
- Maximum 100 service accounts per user (LimitExceededException)
- Maximum 1000 character description length (InvalidInputException)

These limits prevent accidental or malicious resource exhaustion while
not impacting legitimate use cases.

* iam: add missing operational limit constants

Added MaxServiceAccountsPerUser and MaxDescriptionLength constants that
were referenced in the previous commit but not defined.

* iam: enforce service account expiration during authentication

CRITICAL SECURITY FIX: Expired service account credentials were not being
rejected during authentication, allowing continued access after expiration.

Changes:
- Added Expiration field to Credential struct
- Populate expiration when loading service accounts from configuration
- Check expiration in all authentication paths (V2 and V4 signatures)
- Return ErrExpiredToken for expired credentials

This ensures expired service accounts are properly rejected at authentication
time, matching AWS IAM behavior and preventing unauthorized access.

* iam: fix error code for expired service account credentials

Use ErrAccessDenied instead of non-existent ErrExpiredToken for expired
service account credentials. This provides appropriate access denial for
expired credentials while maintaining AWS-compatible error responses.

* iam: fix remaining ErrExpiredToken references

Replace all remaining instances of non-existent ErrExpiredToken with
ErrAccessDenied for expired service account credentials.

* iam: apply AWS-standard key format to user access keys

Updated CreateAccessKey to generate AWS-standard 20-character access keys
with AKIA prefix for regular users, matching the format used for service
accounts. This ensures consistency across all access key types and full
AWS compatibility.

- Access keys: AKIA + 16 random chars = 20 total (was 21 chars, no prefix)
- Secret keys: 40 random chars (was 42, now matches AWS standard)
- Uses AccessKeyLength and UserAccessKeyPrefix constants

* sts: replace fragile string-based error matching with typed errors

Implemented robust error handling using typed errors and errors.Is() instead
of fragile strings.Contains() matching. This decouples the HTTP layer from
service implementation details and prevents errors from being miscategorized
if error messages change.

Changes:
- Added typed error variables to weed/iam/sts/constants.go:
  * ErrTypedTokenExpired
  * ErrTypedInvalidToken
  * ErrTypedInvalidIssuer
  * ErrTypedInvalidAudience
  * ErrTypedMissingClaims

- Updated STS service to wrap provider authentication errors with typed errors
- Replaced strings.Contains() with errors.Is() in HTTP layer for error checking
- Removed TODO comment as the improvement is now implemented

This makes error handling more maintainable and reliable.

* sts: eliminate all string-based error matching with provider-level typed errors

Completed the typed error implementation by adding provider-level typed errors
and updating provider implementations to return them. This eliminates ALL
fragile string matching throughout the entire error handling stack.

Changes:
- Added typed error definitions to weed/iam/providers/errors.go:
  * ErrProviderTokenExpired
  * ErrProviderInvalidToken
  * ErrProviderInvalidIssuer
  * ErrProviderInvalidAudience
  * ErrProviderMissingClaims

- Updated OIDC provider to wrap JWT validation errors with typed provider errors
- Replaced strings.Contains() with errors.Is() in STS service for error mapping
- Complete error chain: Provider -> STS -> HTTP layer, all using errors.Is()

This provides:
- Reliable error classification independent of error message content
- Type-safe error checking throughout the stack
- No order-dependent string matching
- Maintainable error handling that won't break with message changes

* oidc: use jwt.ErrTokenExpired instead of string matching

Replaced the last remaining string-based error check with the JWT library's
exported typed error. This makes the error detection independent of error
message content and more robust against library updates.

Changed from:
  strings.Contains(errMsg, "expired")
To:
  errors.Is(err, jwt.ErrTokenExpired)

This completes the elimination of ALL string-based error matching throughout
the entire authentication stack.

* iam: add description length validation to UpdateServiceAccount

Fixed inconsistency where UpdateServiceAccount didn't validate description
length against MaxDescriptionLength, allowing operational limits to be
bypassed during updates.

Now validates that updated descriptions don't exceed 1000 characters,
matching the validation in CreateServiceAccount.

* iam: refactor expiration check into helper method

Extracted duplicated credential expiration check logic into a helper method
to reduce code duplication and improve maintainability.

Added Credential.isCredentialExpired() method and replaced 5 instances of
inline expiration checks across auth_signature_v2.go and auth_signature_v4.go.

* iam: address critical Copilot security and consistency feedback

Fixed three critical issues identified by Copilot code review:

1. SECURITY: Prevent loading disabled service account credentials
   - Added check to skip disabled service accounts during credential loading
   - Disabled accounts can no longer authenticate

2. Add DurationSeconds validation for STS AssumeRoleWithWebIdentity
   - Enforce AWS-compatible range: 900-43200 seconds (15 min - 12 hours)
   - Returns proper error for out-of-range values

3. Fix expiration update consistency in UpdateServiceAccount
   - Added key existence check like Description field
   - Allows explicit clearing of expiration by setting to empty string
   - Distinguishes between "not updating" and "clearing expiration"

* sts: remove unused durationSecondsStr variable

Fixed build error from unused variable after refactoring duration parsing.

* iam: address remaining Copilot feedback and remove dead code

Completed remaining Copilot code review items:

1. Remove unused getPermission() method (dead code)
   - Method was defined but never called anywhere

2. Improve slice modification safety in DeleteServiceAccount
   - Replaced append-with-slice-operations with filter pattern
   - Avoids potential issues from mutating slice during iteration

3. Fix route registration order
   - Moved STS route registration BEFORE IAM route
   - Prevents IAM route from intercepting STS requests
   - More specific route (with query parameter) now registered first

* iam: improve expiration validation and test cleanup robustness

Addressed additional Copilot feedback:

1. Make expiration validation more explicit
   - Added explicit check for negative values
   - Added comment clarifying that 0 is allowed to clear expiration
   - Improves code readability and intent

2. Fix test cleanup order in s3_service_account_test.go
   - Track created service accounts in a slice
   - Delete all service accounts before deleting parent user
   - Prevents DeleteConflictException during cleanup
   - More robust cleanup even if test fails mid-execution

Note: s3_service_account_security_test.go already had correct cleanup
order due to LIFO defer execution.

* test: remove redundant variable assignments

Removed duplicate assignments of createdSAId, createdAccessKeyId, and
createdSecretAccessKey on lines 148-150 that were already assigned on
lines 132-134.
2025-12-29 20:17:23 -08:00

943 lines
32 KiB
Go

package s3api
import (
"context"
"encoding/json"
"fmt"
"net/http"
"os"
"slices"
"strings"
"sync"
"time"
"github.com/seaweedfs/seaweedfs/weed/credential"
"github.com/seaweedfs/seaweedfs/weed/filer"
"github.com/seaweedfs/seaweedfs/weed/glog"
"github.com/seaweedfs/seaweedfs/weed/kms"
"github.com/seaweedfs/seaweedfs/weed/pb"
"github.com/seaweedfs/seaweedfs/weed/pb/filer_pb"
"github.com/seaweedfs/seaweedfs/weed/pb/iam_pb"
"github.com/seaweedfs/seaweedfs/weed/s3api/s3_constants"
"github.com/seaweedfs/seaweedfs/weed/s3api/s3err"
// Import KMS providers to register them
_ "github.com/seaweedfs/seaweedfs/weed/kms/aws"
// _ "github.com/seaweedfs/seaweedfs/weed/kms/azure" // TODO: Fix Azure SDK compatibility issues
_ "github.com/seaweedfs/seaweedfs/weed/kms/gcp"
_ "github.com/seaweedfs/seaweedfs/weed/kms/local"
_ "github.com/seaweedfs/seaweedfs/weed/kms/openbao"
"google.golang.org/grpc"
)
type Action string
type Iam interface {
Check(f http.HandlerFunc, actions ...Action) http.HandlerFunc
}
type IdentityAccessManagement struct {
m sync.RWMutex
identities []*Identity
accessKeyIdent map[string]*Identity
nameToIdentity map[string]*Identity // O(1) lookup by identity name
accounts map[string]*Account
emailAccount map[string]*Account
hashes map[string]*sync.Pool
hashCounters map[string]*int32
identityAnonymous *Identity
hashMu sync.RWMutex
domain string
isAuthEnabled bool
credentialManager *credential.CredentialManager
filerClient filer_pb.SeaweedFilerClient
grpcDialOption grpc.DialOption
// IAM Integration for advanced features
iamIntegration *S3IAMIntegration
// Bucket policy engine for evaluating bucket policies
policyEngine *BucketPolicyEngine
}
type Identity struct {
Name string
Account *Account
Credentials []*Credential
Actions []Action
PrincipalArn string // ARN for IAM authorization (e.g., "arn:aws:iam::account-id:user/username")
Disabled bool // User status: false = enabled (default), true = disabled
}
// Account represents a system user, a system user can
// configure multiple IAM-Users, IAM-Users can configure
// permissions respectively, and each IAM-User can
// configure multiple security credentials
type Account struct {
//Name is also used to display the "DisplayName" as the owner of the bucket or object
DisplayName string
EmailAddress string
//Id is used to identify an Account when granting cross-account access(ACLs) to buckets and objects
Id string
}
// Predefined Accounts
var (
// AccountAdmin is used as the default account for IAM-Credentials access without Account configured
AccountAdmin = Account{
DisplayName: "admin",
EmailAddress: "admin@example.com",
Id: s3_constants.AccountAdminId,
}
// AccountAnonymous is used to represent the account for anonymous access
AccountAnonymous = Account{
DisplayName: "anonymous",
EmailAddress: "anonymous@example.com",
Id: s3_constants.AccountAnonymousId,
}
)
type Credential struct {
AccessKey string
SecretKey string
Status string // Access key status: "Active" or "Inactive" (empty treated as "Active")
Expiration int64 // Unix timestamp when credential expires (0 = no expiration)
}
// isCredentialExpired checks if a credential has expired
func (c *Credential) isCredentialExpired() bool {
return c.Expiration > 0 && c.Expiration < time.Now().Unix()
}
func NewIdentityAccessManagement(option *S3ApiServerOption) *IdentityAccessManagement {
return NewIdentityAccessManagementWithStore(option, "")
}
func NewIdentityAccessManagementWithStore(option *S3ApiServerOption, explicitStore string) *IdentityAccessManagement {
iam := &IdentityAccessManagement{
domain: option.DomainName,
hashes: make(map[string]*sync.Pool),
hashCounters: make(map[string]*int32),
}
// Always initialize credential manager with fallback to defaults
credentialManager, err := credential.NewCredentialManagerWithDefaults(credential.CredentialStoreTypeName(explicitStore))
if err != nil {
glog.Fatalf("failed to initialize credential manager: %v", err)
}
// For stores that need filer client details, set them temporarily
// This will be updated to use FilerClient's GetCurrentFiler after FilerClient is created
if store := credentialManager.GetStore(); store != nil {
if filerFuncSetter, ok := store.(interface {
SetFilerAddressFunc(func() pb.ServerAddress, grpc.DialOption)
}); ok {
// Temporary setup: use first filer until FilerClient is available
// See s3api_server.go where this is updated to FilerClient.GetCurrentFiler
if len(option.Filers) > 0 {
getFiler := func() pb.ServerAddress {
if len(option.Filers) > 0 {
return option.Filers[0]
}
return ""
}
filerFuncSetter.SetFilerAddressFunc(getFiler, option.GrpcDialOption)
glog.V(1).Infof("Credential store configured with temporary filer function (will be updated after FilerClient creation)")
}
}
}
iam.credentialManager = credentialManager
// Track whether any configuration was successfully loaded
configLoaded := false
// First, try to load configurations from file or filer
if option.Config != "" {
glog.V(3).Infof("loading static config file %s", option.Config)
if err := iam.loadS3ApiConfigurationFromFile(option.Config); err != nil {
glog.Fatalf("fail to load config file %s: %v", option.Config, err)
}
// Check if any identities were actually loaded from the config file
iam.m.RLock()
configLoaded = len(iam.identities) > 0
iam.m.RUnlock()
} else {
glog.V(3).Infof("no static config file specified... loading config from credential manager")
if err := iam.loadS3ApiConfigurationFromFiler(option); err != nil {
glog.Warningf("fail to load config: %v", err)
} else {
// Check if any identities were actually loaded from filer
iam.m.RLock()
configLoaded = len(iam.identities) > 0
iam.m.RUnlock()
}
}
// Only use environment variables as fallback if no configuration was loaded
if !configLoaded {
accessKeyId := os.Getenv("AWS_ACCESS_KEY_ID")
secretAccessKey := os.Getenv("AWS_SECRET_ACCESS_KEY")
if accessKeyId != "" && secretAccessKey != "" {
glog.V(1).Infof("No S3 configuration found, using AWS environment variables as fallback")
// Create environment variable identity name
identityNameSuffix := accessKeyId
if len(accessKeyId) > 8 {
identityNameSuffix = accessKeyId[:8]
}
// Create admin identity with environment variable credentials
envIdentity := &Identity{
Name: "admin-" + identityNameSuffix,
Account: &AccountAdmin,
Credentials: []*Credential{
{
AccessKey: accessKeyId,
SecretKey: secretAccessKey,
},
},
Actions: []Action{
s3_constants.ACTION_ADMIN,
},
}
// Set as the only configuration
iam.m.Lock()
if len(iam.identities) == 0 {
iam.identities = []*Identity{envIdentity}
iam.accessKeyIdent = map[string]*Identity{accessKeyId: envIdentity}
iam.nameToIdentity = map[string]*Identity{envIdentity.Name: envIdentity}
iam.isAuthEnabled = true
}
iam.m.Unlock()
glog.V(1).Infof("Added admin identity from AWS environment variables: %s", envIdentity.Name)
}
}
return iam
}
func (iam *IdentityAccessManagement) loadS3ApiConfigurationFromFiler(option *S3ApiServerOption) error {
return iam.LoadS3ApiConfigurationFromCredentialManager()
}
func (iam *IdentityAccessManagement) loadS3ApiConfigurationFromFile(fileName string) error {
content, readErr := os.ReadFile(fileName)
if readErr != nil {
glog.Warningf("fail to read %s : %v", fileName, readErr)
return fmt.Errorf("fail to read %s : %v", fileName, readErr)
}
// Initialize KMS if configuration contains KMS settings
if err := iam.initializeKMSFromConfig(content); err != nil {
glog.Warningf("KMS initialization failed: %v", err)
}
return iam.LoadS3ApiConfigurationFromBytes(content)
}
func (iam *IdentityAccessManagement) LoadS3ApiConfigurationFromBytes(content []byte) error {
s3ApiConfiguration := &iam_pb.S3ApiConfiguration{}
if err := filer.ParseS3ConfigurationFromBytes(content, s3ApiConfiguration); err != nil {
glog.Warningf("unmarshal error: %v", err)
return fmt.Errorf("unmarshal error: %w", err)
}
if err := filer.CheckDuplicateAccessKey(s3ApiConfiguration); err != nil {
return err
}
if err := iam.loadS3ApiConfiguration(s3ApiConfiguration); err != nil {
return err
}
return nil
}
func (iam *IdentityAccessManagement) loadS3ApiConfiguration(config *iam_pb.S3ApiConfiguration) error {
var identities []*Identity
var identityAnonymous *Identity
accessKeyIdent := make(map[string]*Identity)
nameToIdentity := make(map[string]*Identity)
accounts := make(map[string]*Account)
emailAccount := make(map[string]*Account)
foundAccountAdmin := false
foundAccountAnonymous := false
for _, account := range config.Accounts {
glog.V(3).Infof("loading account name=%s, id=%s", account.DisplayName, account.Id)
switch account.Id {
case AccountAdmin.Id:
AccountAdmin = Account{
Id: account.Id,
DisplayName: account.DisplayName,
EmailAddress: account.EmailAddress,
}
accounts[account.Id] = &AccountAdmin
foundAccountAdmin = true
case AccountAnonymous.Id:
AccountAnonymous = Account{
Id: account.Id,
DisplayName: account.DisplayName,
EmailAddress: account.EmailAddress,
}
accounts[account.Id] = &AccountAnonymous
foundAccountAnonymous = true
default:
t := Account{
Id: account.Id,
DisplayName: account.DisplayName,
EmailAddress: account.EmailAddress,
}
accounts[account.Id] = &t
}
if account.EmailAddress != "" {
emailAccount[account.EmailAddress] = accounts[account.Id]
}
}
if !foundAccountAdmin {
accounts[AccountAdmin.Id] = &AccountAdmin
emailAccount[AccountAdmin.EmailAddress] = &AccountAdmin
}
if !foundAccountAnonymous {
accounts[AccountAnonymous.Id] = &AccountAnonymous
emailAccount[AccountAnonymous.EmailAddress] = &AccountAnonymous
}
for _, ident := range config.Identities {
glog.V(3).Infof("loading identity %s (disabled=%v)", ident.Name, ident.Disabled)
t := &Identity{
Name: ident.Name,
Credentials: nil,
Actions: nil,
PrincipalArn: generatePrincipalArn(ident.Name),
Disabled: ident.Disabled, // false (default) = enabled, true = disabled
}
switch {
case ident.Name == AccountAnonymous.Id:
t.Account = &AccountAnonymous
identityAnonymous = t
case ident.Account == nil:
t.Account = &AccountAdmin
default:
if account, ok := accounts[ident.Account.Id]; ok {
t.Account = account
} else {
t.Account = &AccountAdmin
glog.Warningf("identity %s is associated with a non exist account ID, the association is invalid", ident.Name)
}
}
for _, action := range ident.Actions {
t.Actions = append(t.Actions, Action(action))
}
for _, cred := range ident.Credentials {
t.Credentials = append(t.Credentials, &Credential{
AccessKey: cred.AccessKey,
SecretKey: cred.SecretKey,
Status: cred.Status, // Load access key status
})
accessKeyIdent[cred.AccessKey] = t
}
identities = append(identities, t)
nameToIdentity[t.Name] = t
}
// Load service accounts and add their credentials to the parent identity
for _, sa := range config.ServiceAccounts {
if sa.Credential == nil {
continue
}
// Skip disabled service accounts - they should not be able to authenticate
if sa.Disabled {
glog.V(3).Infof("Skipping disabled service account %s", sa.Id)
continue
}
// Find the parent identity
parentIdent, ok := nameToIdentity[sa.ParentUser]
if !ok {
glog.Warningf("Service account %s has non-existent parent user %s, skipping", sa.Id, sa.ParentUser)
continue
}
// Add service account credential to parent identity with expiration
cred := &Credential{
AccessKey: sa.Credential.AccessKey,
SecretKey: sa.Credential.SecretKey,
Status: sa.Credential.Status,
Expiration: sa.Expiration, // Populate expiration from service account
}
parentIdent.Credentials = append(parentIdent.Credentials, cred)
accessKeyIdent[sa.Credential.AccessKey] = parentIdent
glog.V(3).Infof("Loaded service account %s for parent %s (expiration: %d)", sa.Id, sa.ParentUser, sa.Expiration)
}
iam.m.Lock()
// atomically switch
iam.identities = identities
iam.identityAnonymous = identityAnonymous
iam.accounts = accounts
iam.emailAccount = emailAccount
iam.accessKeyIdent = accessKeyIdent
iam.nameToIdentity = nameToIdentity
if !iam.isAuthEnabled { // one-directional, no toggling
iam.isAuthEnabled = len(identities) > 0
}
iam.m.Unlock()
// Log configuration summary
glog.V(1).Infof("Loaded %d identities, %d accounts, %d access keys. Auth enabled: %v",
len(identities), len(accounts), len(accessKeyIdent), iam.isAuthEnabled)
if glog.V(2) {
glog.V(2).Infof("Access key to identity mapping:")
for accessKey, identity := range accessKeyIdent {
glog.V(2).Infof(" %s -> %s (actions: %d)", accessKey, identity.Name, len(identity.Actions))
}
}
return nil
}
func (iam *IdentityAccessManagement) isEnabled() bool {
return iam.isAuthEnabled
}
func (iam *IdentityAccessManagement) lookupByAccessKey(accessKey string) (identity *Identity, cred *Credential, found bool) {
iam.m.RLock()
defer iam.m.RUnlock()
// Helper function to truncate access key for logging to avoid credential exposure
truncate := func(key string) string {
const mask = "***"
if len(key) > 4 {
return key[:4] + mask
}
// For very short keys, never log the full key
return mask
}
truncatedKey := truncate(accessKey)
glog.V(3).Infof("Looking up access key: %s (len=%d, total keys registered: %d)",
truncatedKey, len(accessKey), len(iam.accessKeyIdent))
if ident, ok := iam.accessKeyIdent[accessKey]; ok {
// Check if user is disabled
if ident.Disabled {
glog.V(2).Infof("User %s is disabled, rejecting access key %s", ident.Name, truncatedKey)
return nil, nil, false
}
for _, credential := range ident.Credentials {
if credential.AccessKey == accessKey {
// Check if access key is inactive (empty Status treated as Active for backward compatibility)
if credential.Status == iamAccessKeyStatusInactive {
glog.V(2).Infof("Access key %s for identity %s is inactive", truncatedKey, ident.Name)
return nil, nil, false
}
glog.V(2).Infof("Found access key %s for identity %s", truncatedKey, ident.Name)
return ident, credential, true
}
}
}
glog.V(2).Infof("Could not find access key %s (len=%d). Available keys: %d, Auth enabled: %v",
truncatedKey, len(accessKey), len(iam.accessKeyIdent), iam.isAuthEnabled)
// Log all registered access keys at higher verbosity for debugging
if glog.V(3) {
glog.V(3).Infof("Registered access keys:")
for key := range iam.accessKeyIdent {
glog.V(3).Infof(" - %s (len=%d)", truncate(key), len(key))
}
}
return nil, nil, false
}
// LookupByAccessKey is an exported wrapper for lookupByAccessKey.
// It returns the identity and credential associated with the given access key.
//
// WARNING: The returned pointers reference internal data structures.
// Callers MUST NOT modify the returned Identity or Credential objects.
// If mutation is needed, make a copy first.
func (iam *IdentityAccessManagement) LookupByAccessKey(accessKey string) (identity *Identity, cred *Credential, found bool) {
return iam.lookupByAccessKey(accessKey)
}
func (iam *IdentityAccessManagement) lookupAnonymous() (identity *Identity, found bool) {
iam.m.RLock()
defer iam.m.RUnlock()
if iam.identityAnonymous != nil {
return iam.identityAnonymous, true
}
return nil, false
}
func (iam *IdentityAccessManagement) lookupByIdentityName(name string) *Identity {
iam.m.RLock()
defer iam.m.RUnlock()
return iam.nameToIdentity[name]
}
// generatePrincipalArn generates an ARN for a user identity
func generatePrincipalArn(identityName string) string {
// Handle special cases
switch identityName {
case AccountAnonymous.Id:
return "arn:aws:iam::user/anonymous"
case AccountAdmin.Id:
return "arn:aws:iam::user/admin"
default:
return fmt.Sprintf("arn:aws:iam::user/%s", identityName)
}
}
func (iam *IdentityAccessManagement) GetAccountNameById(canonicalId string) string {
iam.m.RLock()
defer iam.m.RUnlock()
if account, ok := iam.accounts[canonicalId]; ok {
return account.DisplayName
}
return ""
}
func (iam *IdentityAccessManagement) GetAccountIdByEmail(email string) string {
iam.m.RLock()
defer iam.m.RUnlock()
if account, ok := iam.emailAccount[email]; ok {
return account.Id
}
return ""
}
func (iam *IdentityAccessManagement) Auth(f http.HandlerFunc, action Action) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
if !iam.isEnabled() {
f(w, r)
return
}
identity, errCode := iam.authRequest(r, action)
glog.V(3).Infof("auth error: %v", errCode)
if errCode == s3err.ErrNone {
// Store the authenticated identity in request context (secure, cannot be spoofed)
if identity != nil && identity.Name != "" {
ctx := s3_constants.SetIdentityNameInContext(r.Context(), identity.Name)
// Also store the full identity object for handlers that need it (e.g., ListBuckets)
// This is especially important for JWT users whose identity is not in the identities list
ctx = s3_constants.SetIdentityInContext(ctx, identity)
r = r.WithContext(ctx)
}
f(w, r)
return
}
s3err.WriteErrorResponse(w, r, errCode)
}
}
// check whether the request has valid access keys
func (iam *IdentityAccessManagement) authRequest(r *http.Request, action Action) (*Identity, s3err.ErrorCode) {
var identity *Identity
var s3Err s3err.ErrorCode
var found bool
var authType string
switch getRequestAuthType(r) {
case authTypeUnknown:
glog.V(3).Infof("unknown auth type")
r.Header.Set(s3_constants.AmzAuthType, "Unknown")
return identity, s3err.ErrAccessDenied
case authTypePresignedV2, authTypeSignedV2:
glog.V(3).Infof("v2 auth type")
identity, s3Err = iam.isReqAuthenticatedV2(r)
authType = "SigV2"
case authTypeStreamingSigned, authTypeSigned, authTypePresigned:
glog.V(3).Infof("v4 auth type")
identity, s3Err = iam.reqSignatureV4Verify(r)
authType = "SigV4"
case authTypePostPolicy:
glog.V(3).Infof("post policy auth type")
r.Header.Set(s3_constants.AmzAuthType, "PostPolicy")
return identity, s3err.ErrNone
case authTypeStreamingUnsigned:
glog.V(3).Infof("unsigned streaming upload")
return identity, s3err.ErrNone
case authTypeJWT:
glog.V(3).Infof("jwt auth type detected, iamIntegration != nil? %t", iam.iamIntegration != nil)
r.Header.Set(s3_constants.AmzAuthType, "Jwt")
if iam.iamIntegration != nil {
identity, s3Err = iam.authenticateJWTWithIAM(r)
authType = "Jwt"
} else {
glog.V(2).Infof("IAM integration is nil, returning ErrNotImplemented")
return identity, s3err.ErrNotImplemented
}
case authTypeAnonymous:
authType = "Anonymous"
if identity, found = iam.lookupAnonymous(); !found {
r.Header.Set(s3_constants.AmzAuthType, authType)
return identity, s3err.ErrAccessDenied
}
default:
return identity, s3err.ErrNotImplemented
}
if len(authType) > 0 {
r.Header.Set(s3_constants.AmzAuthType, authType)
}
if s3Err != s3err.ErrNone {
return identity, s3Err
}
glog.V(3).Infof("user name: %v actions: %v, action: %v", identity.Name, identity.Actions, action)
bucket, object := s3_constants.GetBucketAndObject(r)
prefix := s3_constants.GetPrefix(r)
// For List operations, use prefix for permission checking if available
if action == s3_constants.ACTION_LIST && object == "" && prefix != "" {
// List operation with prefix - check permission for the prefix path
object = prefix
} else if (object == "/" || object == "") && prefix != "" {
// Using the aws cli with s3, and s3api, and with boto3, the object is often set to "/" or empty
// but the prefix is set to the actual object key for permission checking
object = prefix
}
// For ListBuckets, authorization is performed in the handler by iterating
// through buckets and checking permissions for each. Skip the global check here.
policyAllows := false
if action == s3_constants.ACTION_LIST && bucket == "" {
// ListBuckets operation - authorization handled per-bucket in the handler
} else {
// First check bucket policy if one exists
// Bucket policies can grant or deny access to specific users/principals
// Following AWS semantics:
// - Explicit DENY in bucket policy → immediate rejection
// - Explicit ALLOW in bucket policy → grant access (bypass IAM checks)
// - No policy or indeterminate → fall through to IAM checks
if iam.policyEngine != nil && bucket != "" {
principal := buildPrincipalARN(identity)
// Phase 1: Evaluate bucket policy without object entry.
// Tag-based conditions (s3:ExistingObjectTag) are re-checked by handlers
// after fetching the entry, which is the Phase 2 check.
allowed, evaluated, err := iam.policyEngine.EvaluatePolicy(bucket, object, string(action), principal, r, nil)
if err != nil {
// SECURITY: Fail-close on policy evaluation errors
// If we can't evaluate the policy, deny access rather than falling through to IAM
glog.Errorf("Error evaluating bucket policy for %s/%s: %v - denying access", bucket, object, err)
return identity, s3err.ErrAccessDenied
} else if evaluated {
// A bucket policy exists and was evaluated with a matching statement
if allowed {
// Policy explicitly allows this action - grant access immediately
// This bypasses IAM checks to support cross-account access and policy-only principals
glog.V(3).Infof("Bucket policy allows %s to %s on %s/%s (bypassing IAM)", identity.Name, action, bucket, object)
policyAllows = true
} else {
// Policy explicitly denies this action - deny access immediately
// Note: Explicit Deny in bucket policy overrides all other permissions
glog.V(3).Infof("Bucket policy explicitly denies %s to %s on %s/%s", identity.Name, action, bucket, object)
return identity, s3err.ErrAccessDenied
}
}
// If not evaluated (no policy or no matching statements), fall through to IAM/identity checks
}
// Only check IAM if bucket policy didn't explicitly allow
if !policyAllows {
// Traditional identities (with Actions from -s3.config) use legacy auth,
// JWT/STS identities (no Actions) use IAM authorization
if len(identity.Actions) > 0 {
if !identity.canDo(action, bucket, object) {
return identity, s3err.ErrAccessDenied
}
} else if iam.iamIntegration != nil {
if errCode := iam.authorizeWithIAM(r, identity, action, bucket, object); errCode != s3err.ErrNone {
return identity, errCode
}
} else {
return identity, s3err.ErrAccessDenied
}
}
}
r.Header.Set(s3_constants.AmzAccountId, identity.Account.Id)
return identity, s3err.ErrNone
}
// AuthSignatureOnly performs only signature verification without any authorization checks.
// This is used for IAM API operations where authorization is handled separately based on
// the specific IAM action (e.g., self-service vs admin operations).
// Returns the authenticated identity and any signature verification error.
func (iam *IdentityAccessManagement) AuthSignatureOnly(r *http.Request) (*Identity, s3err.ErrorCode) {
var identity *Identity
var s3Err s3err.ErrorCode
var authType string
switch getRequestAuthType(r) {
case authTypeUnknown:
glog.V(3).Infof("unknown auth type")
r.Header.Set(s3_constants.AmzAuthType, "Unknown")
return identity, s3err.ErrAccessDenied
case authTypePresignedV2, authTypeSignedV2:
glog.V(3).Infof("v2 auth type")
identity, s3Err = iam.isReqAuthenticatedV2(r)
authType = "SigV2"
case authTypeStreamingSigned, authTypeSigned, authTypePresigned:
glog.V(3).Infof("v4 auth type")
identity, s3Err = iam.reqSignatureV4Verify(r)
authType = "SigV4"
case authTypePostPolicy:
glog.V(3).Infof("post policy auth type")
r.Header.Set(s3_constants.AmzAuthType, "PostPolicy")
return identity, s3err.ErrNone
case authTypeStreamingUnsigned:
glog.V(3).Infof("unsigned streaming upload")
return identity, s3err.ErrNone
case authTypeJWT:
glog.V(3).Infof("jwt auth type detected, iamIntegration != nil? %t", iam.iamIntegration != nil)
r.Header.Set(s3_constants.AmzAuthType, "Jwt")
if iam.iamIntegration != nil {
identity, s3Err = iam.authenticateJWTWithIAM(r)
authType = "Jwt"
} else {
glog.V(2).Infof("IAM integration is nil, returning ErrNotImplemented")
return identity, s3err.ErrNotImplemented
}
case authTypeAnonymous:
// Anonymous users cannot use IAM API
return identity, s3err.ErrAccessDenied
default:
return identity, s3err.ErrNotImplemented
}
if len(authType) > 0 {
r.Header.Set(s3_constants.AmzAuthType, authType)
}
if s3Err != s3err.ErrNone {
return identity, s3Err
}
// Set account ID header for downstream handlers
if identity != nil && identity.Account != nil {
r.Header.Set(s3_constants.AmzAccountId, identity.Account.Id)
}
return identity, s3err.ErrNone
}
func (identity *Identity) canDo(action Action, bucket string, objectKey string) bool {
if identity.isAdmin() {
return true
}
for _, a := range identity.Actions {
// Case where the Resource provided is
// "Resource": [
// "arn:aws:s3:::*"
// ]
if a == action {
return true
}
}
if bucket == "" {
glog.V(3).Infof("identity %s is not allowed to perform action %s on %s -- bucket is empty", identity.Name, action, bucket+objectKey)
return false
}
glog.V(3).Infof("checking if %s can perform %s on bucket '%s'", identity.Name, action, bucket+objectKey)
target := string(action) + ":" + bucket + objectKey
adminTarget := s3_constants.ACTION_ADMIN + ":" + bucket + objectKey
limitedByBucket := string(action) + ":" + bucket
adminLimitedByBucket := s3_constants.ACTION_ADMIN + ":" + bucket
for _, a := range identity.Actions {
act := string(a)
if strings.HasSuffix(act, "*") {
if strings.HasPrefix(target, act[:len(act)-1]) {
return true
}
if strings.HasPrefix(adminTarget, act[:len(act)-1]) {
return true
}
} else {
if act == limitedByBucket {
return true
}
if act == adminLimitedByBucket {
return true
}
}
}
//log error
glog.V(3).Infof("identity %s is not allowed to perform action %s on %s", identity.Name, action, bucket+objectKey)
return false
}
func (identity *Identity) isAdmin() bool {
return slices.Contains(identity.Actions, s3_constants.ACTION_ADMIN)
}
// buildPrincipalARN builds an ARN for an identity to use in bucket policy evaluation
func buildPrincipalARN(identity *Identity) string {
if identity == nil {
return "*" // Anonymous
}
// Check if this is the anonymous user identity (authenticated as anonymous)
// S3 policies expect Principal: "*" for anonymous access
if identity.Name == s3_constants.AccountAnonymousId ||
(identity.Account != nil && identity.Account.Id == s3_constants.AccountAnonymousId) {
return "*" // Anonymous user
}
// Build an AWS-compatible principal ARN
// Format: arn:aws:iam::account-id:user/user-name
accountId := identity.Account.Id
if accountId == "" {
accountId = "000000000000" // Default account ID
}
userName := identity.Name
if userName == "" {
userName = "unknown"
}
return fmt.Sprintf("arn:aws:iam::%s:user/%s", accountId, userName)
}
// GetCredentialManager returns the credential manager instance
func (iam *IdentityAccessManagement) GetCredentialManager() *credential.CredentialManager {
return iam.credentialManager
}
// LoadS3ApiConfigurationFromCredentialManager loads configuration using the credential manager
func (iam *IdentityAccessManagement) LoadS3ApiConfigurationFromCredentialManager() error {
glog.V(1).Infof("Loading S3 API configuration from credential manager")
s3ApiConfiguration, err := iam.credentialManager.LoadConfiguration(context.Background())
if err != nil {
glog.Errorf("Failed to load configuration from credential manager: %v", err)
return fmt.Errorf("failed to load configuration from credential manager: %w", err)
}
glog.V(2).Infof("Credential manager returned %d identities and %d accounts",
len(s3ApiConfiguration.Identities), len(s3ApiConfiguration.Accounts))
if err := iam.loadS3ApiConfiguration(s3ApiConfiguration); err != nil {
glog.Errorf("Failed to load S3 API configuration: %v", err)
return err
}
glog.V(1).Infof("Successfully loaded S3 API configuration from credential manager")
return nil
}
// initializeKMSFromConfig loads KMS configuration from TOML format
func (iam *IdentityAccessManagement) initializeKMSFromConfig(configContent []byte) error {
// JSON-only KMS configuration
if err := iam.initializeKMSFromJSON(configContent); err == nil {
glog.V(1).Infof("Successfully loaded KMS configuration from JSON format")
return nil
}
glog.V(2).Infof("No KMS configuration found in S3 config - SSE-KMS will not be available")
return nil
}
// initializeKMSFromJSON loads KMS configuration from JSON format when provided in the same file
func (iam *IdentityAccessManagement) initializeKMSFromJSON(configContent []byte) error {
// Parse as generic JSON and extract optional "kms" block
var m map[string]any
if err := json.Unmarshal([]byte(strings.TrimSpace(string(configContent))), &m); err != nil {
return err
}
kmsVal, ok := m["kms"]
if !ok {
return fmt.Errorf("no KMS section found")
}
// Load KMS configuration directly from the parsed JSON data
return kms.LoadKMSFromConfig(kmsVal)
}
// SetIAMIntegration sets the IAM integration for advanced authentication and authorization
func (iam *IdentityAccessManagement) SetIAMIntegration(integration *S3IAMIntegration) {
iam.m.Lock()
defer iam.m.Unlock()
iam.iamIntegration = integration
// When IAM integration is configured, authentication must be enabled
// to ensure requests go through proper auth checks
if integration != nil {
iam.isAuthEnabled = true
}
}
// authenticateJWTWithIAM authenticates JWT tokens using the IAM integration
func (iam *IdentityAccessManagement) authenticateJWTWithIAM(r *http.Request) (*Identity, s3err.ErrorCode) {
ctx := r.Context()
// Use IAM integration to authenticate JWT
iamIdentity, errCode := iam.iamIntegration.AuthenticateJWT(ctx, r)
if errCode != s3err.ErrNone {
return nil, errCode
}
// Convert IAMIdentity to existing Identity structure
identity := &Identity{
Name: iamIdentity.Name,
Account: iamIdentity.Account,
Actions: []Action{}, // Empty - authorization handled by policy engine
}
// Store session info in request headers for later authorization
r.Header.Set("X-SeaweedFS-Session-Token", iamIdentity.SessionToken)
r.Header.Set("X-SeaweedFS-Principal", iamIdentity.Principal)
return identity, s3err.ErrNone
}
// authorizeWithIAM authorizes requests using the IAM integration policy engine
func (iam *IdentityAccessManagement) authorizeWithIAM(r *http.Request, identity *Identity, action Action, bucket string, object string) s3err.ErrorCode {
ctx := r.Context()
// Get session info from request headers (for JWT-based authentication)
sessionToken := r.Header.Get("X-SeaweedFS-Session-Token")
principal := r.Header.Get("X-SeaweedFS-Principal")
// Create IAMIdentity for authorization
iamIdentity := &IAMIdentity{
Name: identity.Name,
Account: identity.Account,
}
// Handle both session-based (JWT) and static-key-based (V4 signature) principals
if sessionToken != "" && principal != "" {
// JWT-based authentication - use session token and principal from headers
iamIdentity.Principal = principal
iamIdentity.SessionToken = sessionToken
glog.V(3).Infof("Using JWT-based IAM authorization for principal: %s", principal)
} else if identity.PrincipalArn != "" {
// V4 signature authentication - use principal ARN from identity
iamIdentity.Principal = identity.PrincipalArn
iamIdentity.SessionToken = "" // No session token for static credentials
glog.V(3).Infof("Using V4 signature IAM authorization for principal: %s", identity.PrincipalArn)
} else {
glog.V(3).Info("No valid principal information for IAM authorization")
return s3err.ErrAccessDenied
}
// Use IAM integration for authorization
return iam.iamIntegration.AuthorizeAction(ctx, iamIdentity, action, bucket, object, r)
}