Files
seaweedFS/weed/s3api/auth_signature_v4.go
Chris Lu ae9a943ef6 IAM: Add Service Account Support (#7744) (#7901)
* iam: add ServiceAccount protobuf schema

Add ServiceAccount message type to iam.proto with support for:
- Unique ID and parent user linkage
- Optional expiration timestamp
- Separate credentials (access key/secret)
- Action restrictions (subset of parent)
- Enable/disable status

This is the first step toward implementing issue #7744
(IAM Service Account Support).

* iam: add service account response types

Add IAM API response types for service account operations:
- ServiceAccountInfo struct for marshaling account details
- CreateServiceAccountResponse
- DeleteServiceAccountResponse
- ListServiceAccountsResponse
- GetServiceAccountResponse
- UpdateServiceAccountResponse

Also add type aliases in iamapi package for backwards compatibility.

Part of issue #7744 (IAM Service Account Support).

* iam: implement service account API handlers

Add CRUD operations for service accounts:
- CreateServiceAccount: Creates service account with ABIA key prefix
- DeleteServiceAccount: Removes service account and parent linkage
- ListServiceAccounts: Lists all or filtered by parent user
- GetServiceAccount: Retrieves service account details
- UpdateServiceAccount: Modifies status, description, expiration

Service accounts inherit parent user's actions by default and
support optional expiration timestamps.

Part of issue #7744 (IAM Service Account Support).

* sts: add AssumeRoleWithWebIdentity HTTP endpoint

Add STS API HTTP endpoint for AWS SDK compatibility:
- Create s3api_sts.go with HTTP handlers matching AWS STS spec
- Support AssumeRoleWithWebIdentity action with JWT token
- Return XML response with temporary credentials (AccessKeyId,
  SecretAccessKey, SessionToken) matching AWS format
- Register STS route at POST /?Action=AssumeRoleWithWebIdentity

This enables AWS SDKs (boto3, AWS CLI, etc.) to obtain temporary
S3 credentials using OIDC/JWT tokens.

Part of issue #7744 (IAM Service Account Support).

* test: add service account and STS integration tests

Add integration tests for new IAM features:

s3_service_account_test.go:
- TestServiceAccountLifecycle: Create, Get, List, Update, Delete
- TestServiceAccountValidation: Error handling for missing params

s3_sts_test.go:
- TestAssumeRoleWithWebIdentityValidation: Parameter validation
- TestAssumeRoleWithWebIdentityWithMockJWT: JWT token handling

Tests skip gracefully when SeaweedFS is not running or when IAM
features are not configured.

Part of issue #7744 (IAM Service Account Support).

* iam: address code review comments

- Add constants for service account ID and key lengths
- Use strconv.ParseInt instead of fmt.Sscanf for better error handling
- Allow clearing descriptions by checking key existence in url.Values
- Replace magic numbers (12, 20, 40) with named constants

Addresses review comments from gemini-code-assist[bot]

* test: add proper error handling in service account tests

Use require.NoError(t, err) for io.ReadAll and xml.Unmarshal
to prevent silent failures and ensure test reliability.

Addresses review comment from gemini-code-assist[bot]

* test: add proper error handling in STS tests

Use require.NoError(t, err) for io.ReadAll and xml.Unmarshal
to prevent silent failures and ensure test reliability.
Repeated this fix throughout the file.

Addresses review comment from gemini-code-assist[bot] in PR #7901.

* iam: address additional code review comments

- Specific error code mapping for STS service errors
- Distinguish between Sender and Receiver error types in STS responses
- Add nil checks for credentials in List/GetServiceAccount
- Validate expiration date is in the future
- Improve integration test error messages (include response body)
- Add credential verification step in service account tests

Addresses remaining review comments from gemini-code-assist[bot] across multiple files.

* iam: fix shared slice reference in service account creation

Copy parent's actions to create an independent slice for the service
account instead of sharing the underlying array. This prevents
unexpected mutations when the parent's actions are modified later.

Addresses review comment from coderabbitai[bot] in PR #7901.

* iam: remove duplicate unused constant

Removed redundant iamServiceAccountKeyPrefix as ServiceAccountKeyPrefix
is already defined and used.

Addresses remaining cleanup task.

* sts: document limitation of string-based error mapping

Added TODO comment explaining that the current string-based error
mapping approach is fragile and should be replaced with typed errors
from the STS service in a future refactoring.

This addresses the architectural concern raised in code review while
deferring the actual implementation to a separate PR to avoid scope
creep in the current service account feature addition.

* iam: fix remaining review issues

- Add future-date validation for expiration in UpdateServiceAccount
- Reorder tests so credential verification happens before deletion
- Fix compilation error by using correct JWT generation methods

Addresses final review comments from coderabbitai[bot].

* iam: fix service account access key length

The access key IDs were incorrectly generated with 24 characters
instead of the AWS-standard 20 characters. This was caused by
generating 20 random characters and then prepending the 4-character
ABIA prefix.

Fixed by subtracting the prefix length from AccessKeyLength, so the
final key is: ABIA (4 chars) + random (16 chars) = 20 chars total.

This ensures compatibility with S3 clients that validate key length.

* test: add comprehensive service account security tests

Added comprehensive integration tests for service account functionality:

- TestServiceAccountS3Access: Verify SA credentials work for S3 operations
- TestServiceAccountExpiration: Test expiration date validation and enforcement
- TestServiceAccountInheritedPermissions: Verify parent-child relationship
- TestServiceAccountAccessKeyFormat: Validate AWS-compatible key format (ABIA prefix, 20 char length)

These tests ensure SeaweedFS service accounts are compatible with AWS
conventions and provide robust security coverage.

* iam: remove unused UserAccessKeyPrefix constant

Code cleanup to remove unused constants.

* iam: remove unused iamCommonResponse type alias

Code cleanup to remove unused type aliases.

* iam: restore and use UserAccessKeyPrefix constant

Restored UserAccessKeyPrefix constant and updated s3api tests to use it
instead of hardcoded strings for better maintainability and consistency.

* test: improve error handling in service account security tests

Added explicit error checking for io.ReadAll and xml.Unmarshal in
TestServiceAccountExpiration to ensure failures are reported correctly and
cleanup is performed only when appropriate. Also added logging for failed
responses.

* test: use t.Cleanup for reliable resource cleanup

Replaced defer with t.Cleanup to ensure service account cleanup runs even
when require.NoError fails. Also switched from manual error checking to
require.NoError for more idiomatic testify usage.

* iam: add CreatedBy field and optimize identity lookups

- Added createdBy parameter to CreateServiceAccount to track who created each service account
- Extract creator identity from request context using GetIdentityNameFromContext
- Populate created_by field in ServiceAccount protobuf
- Added findIdentityByName helper function to optimize identity lookups
- Replaced nested loops with O(n) helper function calls in CreateServiceAccount and DeleteServiceAccount

This addresses code review feedback for better auditing and performance.

* iam: prevent user deletion when service accounts exist

Following AWS IAM behavior, prevent deletion of users that have active
service accounts. This ensures explicit cleanup and prevents orphaned
service account resources with invalid ParentUser references.

Users must delete all associated service accounts before deleting the
parent user, providing safer resource management.

* sts: enhance TODO with typed error implementation guidance

Updated TODO comment with detailed implementation approach for replacing
string-based error matching with typed errors using errors.Is(). This
provides a clear roadmap for a follow-up PR to improve error handling
robustness and maintainability.

* iam: add operational limits for service account creation

Added AWS IAM-compatible safeguards to prevent resource exhaustion:
- Maximum 100 service accounts per user (LimitExceededException)
- Maximum 1000 character description length (InvalidInputException)

These limits prevent accidental or malicious resource exhaustion while
not impacting legitimate use cases.

* iam: add missing operational limit constants

Added MaxServiceAccountsPerUser and MaxDescriptionLength constants that
were referenced in the previous commit but not defined.

* iam: enforce service account expiration during authentication

CRITICAL SECURITY FIX: Expired service account credentials were not being
rejected during authentication, allowing continued access after expiration.

Changes:
- Added Expiration field to Credential struct
- Populate expiration when loading service accounts from configuration
- Check expiration in all authentication paths (V2 and V4 signatures)
- Return ErrExpiredToken for expired credentials

This ensures expired service accounts are properly rejected at authentication
time, matching AWS IAM behavior and preventing unauthorized access.

* iam: fix error code for expired service account credentials

Use ErrAccessDenied instead of non-existent ErrExpiredToken for expired
service account credentials. This provides appropriate access denial for
expired credentials while maintaining AWS-compatible error responses.

* iam: fix remaining ErrExpiredToken references

Replace all remaining instances of non-existent ErrExpiredToken with
ErrAccessDenied for expired service account credentials.

* iam: apply AWS-standard key format to user access keys

Updated CreateAccessKey to generate AWS-standard 20-character access keys
with AKIA prefix for regular users, matching the format used for service
accounts. This ensures consistency across all access key types and full
AWS compatibility.

- Access keys: AKIA + 16 random chars = 20 total (was 21 chars, no prefix)
- Secret keys: 40 random chars (was 42, now matches AWS standard)
- Uses AccessKeyLength and UserAccessKeyPrefix constants

* sts: replace fragile string-based error matching with typed errors

Implemented robust error handling using typed errors and errors.Is() instead
of fragile strings.Contains() matching. This decouples the HTTP layer from
service implementation details and prevents errors from being miscategorized
if error messages change.

Changes:
- Added typed error variables to weed/iam/sts/constants.go:
  * ErrTypedTokenExpired
  * ErrTypedInvalidToken
  * ErrTypedInvalidIssuer
  * ErrTypedInvalidAudience
  * ErrTypedMissingClaims

- Updated STS service to wrap provider authentication errors with typed errors
- Replaced strings.Contains() with errors.Is() in HTTP layer for error checking
- Removed TODO comment as the improvement is now implemented

This makes error handling more maintainable and reliable.

* sts: eliminate all string-based error matching with provider-level typed errors

Completed the typed error implementation by adding provider-level typed errors
and updating provider implementations to return them. This eliminates ALL
fragile string matching throughout the entire error handling stack.

Changes:
- Added typed error definitions to weed/iam/providers/errors.go:
  * ErrProviderTokenExpired
  * ErrProviderInvalidToken
  * ErrProviderInvalidIssuer
  * ErrProviderInvalidAudience
  * ErrProviderMissingClaims

- Updated OIDC provider to wrap JWT validation errors with typed provider errors
- Replaced strings.Contains() with errors.Is() in STS service for error mapping
- Complete error chain: Provider -> STS -> HTTP layer, all using errors.Is()

This provides:
- Reliable error classification independent of error message content
- Type-safe error checking throughout the stack
- No order-dependent string matching
- Maintainable error handling that won't break with message changes

* oidc: use jwt.ErrTokenExpired instead of string matching

Replaced the last remaining string-based error check with the JWT library's
exported typed error. This makes the error detection independent of error
message content and more robust against library updates.

Changed from:
  strings.Contains(errMsg, "expired")
To:
  errors.Is(err, jwt.ErrTokenExpired)

This completes the elimination of ALL string-based error matching throughout
the entire authentication stack.

* iam: add description length validation to UpdateServiceAccount

Fixed inconsistency where UpdateServiceAccount didn't validate description
length against MaxDescriptionLength, allowing operational limits to be
bypassed during updates.

Now validates that updated descriptions don't exceed 1000 characters,
matching the validation in CreateServiceAccount.

* iam: refactor expiration check into helper method

Extracted duplicated credential expiration check logic into a helper method
to reduce code duplication and improve maintainability.

Added Credential.isCredentialExpired() method and replaced 5 instances of
inline expiration checks across auth_signature_v2.go and auth_signature_v4.go.

* iam: address critical Copilot security and consistency feedback

Fixed three critical issues identified by Copilot code review:

1. SECURITY: Prevent loading disabled service account credentials
   - Added check to skip disabled service accounts during credential loading
   - Disabled accounts can no longer authenticate

2. Add DurationSeconds validation for STS AssumeRoleWithWebIdentity
   - Enforce AWS-compatible range: 900-43200 seconds (15 min - 12 hours)
   - Returns proper error for out-of-range values

3. Fix expiration update consistency in UpdateServiceAccount
   - Added key existence check like Description field
   - Allows explicit clearing of expiration by setting to empty string
   - Distinguishes between "not updating" and "clearing expiration"

* sts: remove unused durationSecondsStr variable

Fixed build error from unused variable after refactoring duration parsing.

* iam: address remaining Copilot feedback and remove dead code

Completed remaining Copilot code review items:

1. Remove unused getPermission() method (dead code)
   - Method was defined but never called anywhere

2. Improve slice modification safety in DeleteServiceAccount
   - Replaced append-with-slice-operations with filter pattern
   - Avoids potential issues from mutating slice during iteration

3. Fix route registration order
   - Moved STS route registration BEFORE IAM route
   - Prevents IAM route from intercepting STS requests
   - More specific route (with query parameter) now registered first

* iam: improve expiration validation and test cleanup robustness

Addressed additional Copilot feedback:

1. Make expiration validation more explicit
   - Added explicit check for negative values
   - Added comment clarifying that 0 is allowed to clear expiration
   - Improves code readability and intent

2. Fix test cleanup order in s3_service_account_test.go
   - Track created service accounts in a slice
   - Delete all service accounts before deleting parent user
   - Prevents DeleteConflictException during cleanup
   - More robust cleanup even if test fails mid-execution

Note: s3_service_account_security_test.go already had correct cleanup
order due to LIFO defer execution.

* test: remove redundant variable assignments

Removed duplicate assignments of createdSAId, createdAccessKeyId, and
createdSecretAccessKey on lines 148-150 that were already assigned on
lines 132-134.
2025-12-29 20:17:23 -08:00

879 lines
28 KiB
Go

/*
* The following code tries to reverse engineer the Amazon S3 APIs,
* and is mostly copied from minio implementation.
*/
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
// implied. See the License for the specific language governing
// permissions and limitations under the License.
package s3api
import (
"bytes"
"crypto/hmac"
"crypto/sha256"
"crypto/subtle"
"encoding/hex"
"io"
"net"
"net/http"
"regexp"
"sort"
"strconv"
"strings"
"time"
"unicode/utf8"
"github.com/seaweedfs/seaweedfs/weed/glog"
"github.com/seaweedfs/seaweedfs/weed/s3api/s3_constants"
"github.com/seaweedfs/seaweedfs/weed/s3api/s3err"
)
func (iam *IdentityAccessManagement) reqSignatureV4Verify(r *http.Request) (*Identity, s3err.ErrorCode) {
switch {
case isRequestSignatureV4(r):
identity, _, errCode := iam.doesSignatureMatch(r)
return identity, errCode
case isRequestPresignedSignatureV4(r):
identity, _, errCode := iam.doesPresignedSignatureMatch(r)
return identity, errCode
}
return nil, s3err.ErrAccessDenied
}
// Constants specific to this file
const (
emptySHA256 = "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
streamingContentSHA256 = "STREAMING-AWS4-HMAC-SHA256-PAYLOAD"
streamingContentSHA256Trailer = "STREAMING-AWS4-HMAC-SHA256-PAYLOAD-TRAILER"
streamingUnsignedPayload = "STREAMING-UNSIGNED-PAYLOAD-TRAILER"
unsignedPayload = "UNSIGNED-PAYLOAD"
// Limit for IAM/STS request body size to prevent DoS attacks
iamRequestBodyLimit = 10 * (1 << 20) // 10 MiB
)
// streamHashRequestBody computes SHA256 hash incrementally while preserving the body.
func streamHashRequestBody(r *http.Request, sizeLimit int64) (string, error) {
if r.Body == nil {
return emptySHA256, nil
}
limitedReader := io.LimitReader(r.Body, sizeLimit)
hasher := sha256.New()
var bodyBuffer bytes.Buffer
// Use io.Copy with an io.MultiWriter to hash and buffer the body simultaneously.
if _, err := io.Copy(io.MultiWriter(hasher, &bodyBuffer), limitedReader); err != nil {
return "", err
}
r.Body = io.NopCloser(&bodyBuffer)
if bodyBuffer.Len() == 0 {
return emptySHA256, nil
}
return hex.EncodeToString(hasher.Sum(nil)), nil
}
// getContentSha256Cksum retrieves the "x-amz-content-sha256" header value.
func getContentSha256Cksum(r *http.Request) string {
// If the client sends a SHA256 checksum of the object in this header, use it.
if v := r.Header.Get("X-Amz-Content-Sha256"); v != "" {
return v
}
// For a presigned request we look at the query param for sha256.
if isRequestPresignedSignatureV4(r) {
// X-Amz-Content-Sha256 header value is optional for presigned requests.
return unsignedPayload
}
// X-Amz-Content-Sha256 header value is required for all non-presigned requests.
return emptySHA256
}
// signValues data type represents structured form of AWS Signature V4 header.
type signValues struct {
Credential credentialHeader
SignedHeaders []string
Signature string
}
// parseSignV4 parses the authorization header for signature v4.
func parseSignV4(v4Auth string) (sv signValues, aec s3err.ErrorCode) {
// Replace all spaced strings, some clients can send spaced
// parameters and some won't. So we pro-actively remove any spaces
// to make parsing easier.
v4Auth = strings.Replace(v4Auth, " ", "", -1)
if v4Auth == "" {
return sv, s3err.ErrAuthHeaderEmpty
}
// Verify if the header algorithm is supported or not.
if !strings.HasPrefix(v4Auth, signV4Algorithm) {
return sv, s3err.ErrSignatureVersionNotSupported
}
// Strip off the Algorithm prefix.
v4Auth = strings.TrimPrefix(v4Auth, signV4Algorithm)
authFields := strings.Split(strings.TrimSpace(v4Auth), ",")
if len(authFields) != 3 {
return sv, s3err.ErrMissingFields
}
// Initialize signature version '4' structured header.
signV4Values := signValues{}
var err s3err.ErrorCode
// Save credential values.
signV4Values.Credential, err = parseCredentialHeader(authFields[0])
if err != s3err.ErrNone {
return sv, err
}
// Save signed headers.
signV4Values.SignedHeaders, err = parseSignedHeader(authFields[1])
if err != s3err.ErrNone {
return sv, err
}
// Save signature.
signV4Values.Signature, err = parseSignature(authFields[2])
if err != s3err.ErrNone {
return sv, err
}
// Return the structure here.
return signV4Values, s3err.ErrNone
}
// buildPathWithForwardedPrefix combines forwarded prefix with URL path while preserving S3 key semantics.
// This function avoids path.Clean which would collapse "//" and dot segments, breaking S3 signatures.
// It only normalizes the join boundary to avoid double slashes between prefix and path.
func buildPathWithForwardedPrefix(forwardedPrefix, urlPath string) string {
if forwardedPrefix == "" {
return urlPath
}
// Ensure single leading slash on prefix
if !strings.HasPrefix(forwardedPrefix, "/") {
forwardedPrefix = "/" + forwardedPrefix
}
// Join without collapsing interior segments; only fix a double slash at the boundary
var joined string
if strings.HasSuffix(forwardedPrefix, "/") && strings.HasPrefix(urlPath, "/") {
joined = forwardedPrefix + urlPath[1:]
} else if !strings.HasSuffix(forwardedPrefix, "/") && !strings.HasPrefix(urlPath, "/") {
joined = forwardedPrefix + "/" + urlPath
} else {
joined = forwardedPrefix + urlPath
}
// Trailing slash semantics inherited from urlPath (already present if needed)
return joined
}
// v4AuthInfo holds the parsed authentication data from a request,
// whether it's from the Authorization header or presigned URL query parameters.
type v4AuthInfo struct {
Signature string
AccessKey string
SignedHeaders []string
Date time.Time
Region string
Service string
Scope string
HashedPayload string
IsPresigned bool
}
// verifyV4Signature is the single entry point for verifying any AWS Signature V4 request.
// It handles standard requests, presigned URLs, and the seed signature for streaming uploads.
func (iam *IdentityAccessManagement) verifyV4Signature(r *http.Request, shouldCheckPermissions bool) (identity *Identity, credential *Credential, calculatedSignature string, authInfo *v4AuthInfo, errCode s3err.ErrorCode) {
// 1. Extract authentication information from header or query parameters
authInfo, errCode = extractV4AuthInfo(r)
if errCode != s3err.ErrNone {
return nil, nil, "", nil, errCode
}
// 2. Lookup user and credentials
identity, cred, found := iam.lookupByAccessKey(authInfo.AccessKey)
if !found {
// Log detailed error information for InvalidAccessKeyId
iam.m.RLock()
availableKeys := make([]string, 0, len(iam.accessKeyIdent))
for key := range iam.accessKeyIdent {
availableKeys = append(availableKeys, key)
}
iam.m.RUnlock()
glog.Warningf("InvalidAccessKeyId: attempted key '%s' not found. Available keys: %d, Auth enabled: %v",
authInfo.AccessKey, len(availableKeys), iam.isAuthEnabled)
if glog.V(2) && len(availableKeys) > 0 {
glog.V(2).Infof("Available access keys: %v", availableKeys)
}
return nil, nil, "", nil, s3err.ErrInvalidAccessKeyID
}
// Check service account expiration
if cred.isCredentialExpired() {
glog.V(2).Infof("Service account credential %s has expired (expiration: %d, now: %d)",
authInfo.AccessKey, cred.Expiration, time.Now().Unix())
return nil, nil, "", nil, s3err.ErrAccessDenied
}
// 3. Perform permission check
if shouldCheckPermissions {
bucket, object := s3_constants.GetBucketAndObject(r)
action := s3_constants.ACTION_READ
if r.Method != http.MethodGet && r.Method != http.MethodHead {
action = s3_constants.ACTION_WRITE
}
if !identity.canDo(Action(action), bucket, object) {
return nil, nil, "", nil, s3err.ErrAccessDenied
}
}
// 4. Handle presigned request expiration
if authInfo.IsPresigned {
if errCode = checkPresignedRequestExpiry(r, authInfo.Date); errCode != s3err.ErrNone {
return nil, nil, "", nil, errCode
}
}
// 5. Extract headers that were part of the signature
extractedSignedHeaders, errCode := extractSignedHeaders(authInfo.SignedHeaders, r)
if errCode != s3err.ErrNone {
return nil, nil, "", nil, errCode
}
// 6. Get the query string for the canonical request
queryStr := getCanonicalQueryString(r, authInfo.IsPresigned)
// 7. Define a closure for the core verification logic to avoid repetition
verify := func(urlPath string) (string, s3err.ErrorCode) {
return calculateAndVerifySignature(
cred.SecretKey,
r.Method,
urlPath,
queryStr,
extractedSignedHeaders,
authInfo,
)
}
// 8. Verify the signature, trying with X-Forwarded-Prefix first
if forwardedPrefix := r.Header.Get("X-Forwarded-Prefix"); forwardedPrefix != "" {
cleanedPath := buildPathWithForwardedPrefix(forwardedPrefix, r.URL.Path)
calculatedSignature, errCode = verify(cleanedPath)
if errCode == s3err.ErrNone {
return identity, cred, calculatedSignature, authInfo, s3err.ErrNone
}
}
// 9. Verify with the original path
calculatedSignature, errCode = verify(r.URL.Path)
if errCode != s3err.ErrNone {
return nil, nil, "", nil, errCode
}
return identity, cred, calculatedSignature, authInfo, s3err.ErrNone
}
// calculateAndVerifySignature contains the core logic for creating the canonical request,
// string-to-sign, and comparing the final signature.
func calculateAndVerifySignature(secretKey, method, urlPath, queryStr string, extractedSignedHeaders http.Header, authInfo *v4AuthInfo) (string, s3err.ErrorCode) {
canonicalRequest := getCanonicalRequest(extractedSignedHeaders, authInfo.HashedPayload, queryStr, urlPath, method)
stringToSign := getStringToSign(canonicalRequest, authInfo.Date, authInfo.Scope)
signingKey := getSigningKey(secretKey, authInfo.Date.Format(yyyymmdd), authInfo.Region, authInfo.Service)
newSignature := getSignature(signingKey, stringToSign)
if !compareSignatureV4(newSignature, authInfo.Signature) {
glog.V(4).Infof("Signature mismatch. Details:\n- CanonicalRequest: %q\n- StringToSign: %q\n- Calculated: %s, Provided: %s",
canonicalRequest, stringToSign, newSignature, authInfo.Signature)
return "", s3err.ErrSignatureDoesNotMatch
}
return newSignature, s3err.ErrNone
}
func extractV4AuthInfo(r *http.Request) (*v4AuthInfo, s3err.ErrorCode) {
if isRequestPresignedSignatureV4(r) {
return extractV4AuthInfoFromQuery(r)
}
return extractV4AuthInfoFromHeader(r)
}
func extractV4AuthInfoFromHeader(r *http.Request) (*v4AuthInfo, s3err.ErrorCode) {
authHeader := r.Header.Get("Authorization")
signV4Values, errCode := parseSignV4(authHeader)
if errCode != s3err.ErrNone {
return nil, errCode
}
var t time.Time
if xamz := r.Header.Get("x-amz-date"); xamz != "" {
parsed, err := time.Parse(iso8601Format, xamz)
if err != nil {
return nil, s3err.ErrMalformedDate
}
t = parsed
} else {
ds := r.Header.Get("Date")
if ds == "" {
return nil, s3err.ErrMissingDateHeader
}
parsed, err := http.ParseTime(ds)
if err != nil {
return nil, s3err.ErrMalformedDate
}
t = parsed.UTC()
}
// Validate clock skew: requests cannot be older than 15 minutes from server time to prevent replay attacks
const maxSkew = 15 * time.Minute
now := time.Now().UTC()
if now.Sub(t) > maxSkew || t.Sub(now) > maxSkew {
return nil, s3err.ErrRequestTimeTooSkewed
}
hashedPayload := getContentSha256Cksum(r)
if signV4Values.Credential.scope.service != "s3" && hashedPayload == emptySHA256 && r.Body != nil {
var hashErr error
hashedPayload, hashErr = streamHashRequestBody(r, iamRequestBodyLimit)
if hashErr != nil {
return nil, s3err.ErrInternalError
}
}
return &v4AuthInfo{
Signature: signV4Values.Signature,
AccessKey: signV4Values.Credential.accessKey,
SignedHeaders: signV4Values.SignedHeaders,
Date: t,
Region: signV4Values.Credential.scope.region,
Service: signV4Values.Credential.scope.service,
Scope: signV4Values.Credential.getScope(),
HashedPayload: hashedPayload,
IsPresigned: false,
}, s3err.ErrNone
}
func extractV4AuthInfoFromQuery(r *http.Request) (*v4AuthInfo, s3err.ErrorCode) {
query := r.URL.Query()
// Validate all required query parameters upfront for fail-fast behavior
if query.Get("X-Amz-Algorithm") != signV4Algorithm {
return nil, s3err.ErrSignatureVersionNotSupported
}
if query.Get("X-Amz-Date") == "" {
return nil, s3err.ErrMissingDateHeader
}
if query.Get("X-Amz-Credential") == "" {
return nil, s3err.ErrMissingFields
}
if query.Get("X-Amz-Signature") == "" {
return nil, s3err.ErrMissingFields
}
if query.Get("X-Amz-SignedHeaders") == "" {
return nil, s3err.ErrMissingFields
}
if query.Get("X-Amz-Expires") == "" {
return nil, s3err.ErrInvalidQueryParams
}
// Parse date
dateStr := query.Get("X-Amz-Date")
t, err := time.Parse(iso8601Format, dateStr)
if err != nil {
return nil, s3err.ErrMalformedDate
}
// Parse credential header
credHeader, errCode := parseCredentialHeader("Credential=" + query.Get("X-Amz-Credential"))
if errCode != s3err.ErrNone {
return nil, errCode
}
// For presigned URLs, X-Amz-Content-Sha256 must come from the query parameter
// (or default to UNSIGNED-PAYLOAD) because that's what was used for signing.
// We must NOT check the request header as it wasn't part of the signature calculation.
hashedPayload := query.Get("X-Amz-Content-Sha256")
if hashedPayload == "" {
hashedPayload = unsignedPayload
}
return &v4AuthInfo{
Signature: query.Get("X-Amz-Signature"),
AccessKey: credHeader.accessKey,
SignedHeaders: strings.Split(query.Get("X-Amz-SignedHeaders"), ";"),
Date: t,
Region: credHeader.scope.region,
Service: credHeader.scope.service,
Scope: credHeader.getScope(),
HashedPayload: hashedPayload,
IsPresigned: true,
}, s3err.ErrNone
}
func getCanonicalQueryString(r *http.Request, isPresigned bool) string {
var queryToEncode string
if !isPresigned {
queryToEncode = r.URL.Query().Encode()
} else {
queryForCanonical := r.URL.Query()
queryForCanonical.Del("X-Amz-Signature")
queryToEncode = queryForCanonical.Encode()
}
return queryToEncode
}
func checkPresignedRequestExpiry(r *http.Request, t time.Time) s3err.ErrorCode {
expiresStr := r.URL.Query().Get("X-Amz-Expires")
// X-Amz-Expires is validated as required in extractV4AuthInfoFromQuery,
// so it should never be empty here
expires, err := strconv.ParseInt(expiresStr, 10, 64)
if err != nil {
return s3err.ErrMalformedDate
}
// The maximum value for X-Amz-Expires is 604800 seconds (7 days)
// Allow 0 but it will immediately fail expiration check
if expires < 0 {
return s3err.ErrNegativeExpires
}
if expires > 604800 {
return s3err.ErrMaximumExpires
}
expirationTime := t.Add(time.Duration(expires) * time.Second)
if time.Now().UTC().After(expirationTime) {
return s3err.ErrExpiredPresignRequest
}
return s3err.ErrNone
}
func (iam *IdentityAccessManagement) doesSignatureMatch(r *http.Request) (*Identity, string, s3err.ErrorCode) {
identity, _, calculatedSignature, _, errCode := iam.verifyV4Signature(r, false)
return identity, calculatedSignature, errCode
}
func (iam *IdentityAccessManagement) doesPresignedSignatureMatch(r *http.Request) (*Identity, string, s3err.ErrorCode) {
identity, _, calculatedSignature, _, errCode := iam.verifyV4Signature(r, false)
return identity, calculatedSignature, errCode
}
// credentialHeader data type represents structured form of Credential
// string from authorization header.
type credentialHeader struct {
accessKey string
scope struct {
date time.Time
region string
service string
request string
}
}
func (c credentialHeader) getScope() string {
return strings.Join([]string{
c.scope.date.Format(yyyymmdd),
c.scope.region,
c.scope.service,
c.scope.request,
}, "/")
}
// parse credentialHeader string into its structured form.
func parseCredentialHeader(credElement string) (ch credentialHeader, aec s3err.ErrorCode) {
creds := strings.SplitN(strings.TrimSpace(credElement), "=", 2)
if len(creds) != 2 {
return ch, s3err.ErrMissingFields
}
if creds[0] != "Credential" {
return ch, s3err.ErrMissingCredTag
}
credElements := strings.Split(strings.TrimSpace(creds[1]), "/")
if len(credElements) != 5 {
return ch, s3err.ErrCredMalformed
}
// Save access key id.
cred := credentialHeader{
accessKey: credElements[0],
}
var e error
cred.scope.date, e = time.Parse(yyyymmdd, credElements[1])
if e != nil {
return ch, s3err.ErrMalformedCredentialDate
}
cred.scope.region = credElements[2]
cred.scope.service = credElements[3] // "s3"
cred.scope.request = credElements[4] // "aws4_request"
return cred, s3err.ErrNone
}
// Parse signature from signature tag.
func parseSignature(signElement string) (string, s3err.ErrorCode) {
signFields := strings.Split(strings.TrimSpace(signElement), "=")
if len(signFields) != 2 {
return "", s3err.ErrMissingFields
}
if signFields[0] != "Signature" {
return "", s3err.ErrMissingSignTag
}
if signFields[1] == "" {
return "", s3err.ErrMissingFields
}
signature := signFields[1]
return signature, s3err.ErrNone
}
// Parse slice of signed headers from signed headers tag.
func parseSignedHeader(signedHdrElement string) ([]string, s3err.ErrorCode) {
signedHdrFields := strings.Split(strings.TrimSpace(signedHdrElement), "=")
if len(signedHdrFields) != 2 {
return nil, s3err.ErrMissingFields
}
if signedHdrFields[0] != "SignedHeaders" {
return nil, s3err.ErrMissingSignHeadersTag
}
if signedHdrFields[1] == "" {
return nil, s3err.ErrMissingFields
}
signedHeaders := strings.Split(signedHdrFields[1], ";")
return signedHeaders, s3err.ErrNone
}
func (iam *IdentityAccessManagement) doesPolicySignatureV4Match(formValues http.Header) s3err.ErrorCode {
// Parse credential tag.
credHeader, err := parseCredentialHeader("Credential=" + formValues.Get("X-Amz-Credential"))
if err != s3err.ErrNone {
return err
}
identity, cred, found := iam.lookupByAccessKey(credHeader.accessKey)
if !found {
// Log detailed error information for InvalidAccessKeyId (POST policy)
iam.m.RLock()
availableKeyCount := len(iam.accessKeyIdent)
iam.m.RUnlock()
glog.Warningf("InvalidAccessKeyId (POST policy): attempted key '%s' not found. Available keys: %d, Auth enabled: %v",
credHeader.accessKey, availableKeyCount, iam.isAuthEnabled)
return s3err.ErrInvalidAccessKeyID
}
// Check service account expiration
if cred.isCredentialExpired() {
glog.V(2).Infof("Service account credential %s has expired (expiration: %d, now: %d)",
credHeader.accessKey, cred.Expiration, time.Now().Unix())
return s3err.ErrAccessDenied
}
bucket := formValues.Get("bucket")
if !identity.canDo(s3_constants.ACTION_WRITE, bucket, "") {
return s3err.ErrAccessDenied
}
// Get signing key.
signingKey := getSigningKey(cred.SecretKey, credHeader.scope.date.Format(yyyymmdd), credHeader.scope.region, credHeader.scope.service)
// Get signature.
newSignature := getSignature(signingKey, formValues.Get("Policy"))
// Verify signature.
if !compareSignatureV4(newSignature, formValues.Get("X-Amz-Signature")) {
return s3err.ErrSignatureDoesNotMatch
}
return s3err.ErrNone
}
// Verify if extracted signed headers are not properly signed.
func extractSignedHeaders(signedHeaders []string, r *http.Request) (http.Header, s3err.ErrorCode) {
reqHeaders := r.Header
// If no signed headers are provided, then return an error.
if len(signedHeaders) == 0 {
return nil, s3err.ErrMissingFields
}
extractedSignedHeaders := make(http.Header)
for _, header := range signedHeaders {
// `host` is not a case-sensitive header, unlike other headers such as `x-amz-date`.
if header == "host" {
// Get host value.
hostHeaderValue := extractHostHeader(r)
extractedSignedHeaders[header] = []string{hostHeaderValue}
continue
}
// For all other headers we need to find them in the HTTP headers and copy them over.
// We skip non-existent headers to be compatible with AWS signatures.
if values, ok := reqHeaders[http.CanonicalHeaderKey(header)]; ok {
extractedSignedHeaders[header] = values
}
}
return extractedSignedHeaders, s3err.ErrNone
}
// extractHostHeader returns the value of host header if available.
func extractHostHeader(r *http.Request) string {
forwardedHost := r.Header.Get("X-Forwarded-Host")
forwardedPort := r.Header.Get("X-Forwarded-Port")
forwardedProto := r.Header.Get("X-Forwarded-Proto")
// Determine the effective scheme with correct order of precedence:
// 1. X-Forwarded-Proto (most authoritative, reflects client's original protocol)
// 2. r.TLS (authoritative for direct connection to server)
// 3. r.URL.Scheme (fallback, may not always be set correctly)
// 4. Default to "http"
scheme := "http"
if r.URL.Scheme != "" {
scheme = r.URL.Scheme
}
if r.TLS != nil {
scheme = "https"
}
if forwardedProto != "" {
scheme = forwardedProto
}
var host, port string
if forwardedHost != "" {
// X-Forwarded-Host can be a comma-separated list of hosts when there are multiple proxies.
// Use only the first host in the list and trim spaces for robustness.
if comma := strings.Index(forwardedHost, ","); comma != -1 {
host = strings.TrimSpace(forwardedHost[:comma])
} else {
host = strings.TrimSpace(forwardedHost)
}
port = forwardedPort
if h, p, err := net.SplitHostPort(host); err == nil {
host = h
if port == "" {
port = p
}
}
} else {
host = r.Host
if host == "" {
host = r.URL.Host
}
if h, p, err := net.SplitHostPort(host); err == nil {
host = h
port = p
}
}
// If we have a non-default port, join it with the host.
// net.JoinHostPort will handle bracketing for IPv6.
if port != "" && !isDefaultPort(scheme, port) {
// Strip existing brackets before calling JoinHostPort, which automatically adds
// brackets for IPv6 addresses. This prevents double-bracketing like [[::1]]:8080.
// Using Trim handles both well-formed and malformed bracketed hosts.
host = strings.Trim(host, "[]")
return net.JoinHostPort(host, port)
}
// No port or default port was stripped. According to AWS SDK behavior (aws-sdk-go-v2),
// when a default port is removed from an IPv6 address, the brackets should also be removed.
// This matches AWS S3 signature calculation requirements.
// Reference: https://github.com/aws/aws-sdk-go-v2/blob/main/aws/signer/internal/v4/host.go
// The stripPort function returns IPv6 without brackets when port is stripped.
if strings.Contains(host, ":") {
// This is an IPv6 address. Strip brackets to match AWS SDK behavior.
return strings.Trim(host, "[]")
}
return host
}
func isDefaultPort(scheme, port string) bool {
if port == "" {
return true
}
switch port {
case "80":
return strings.EqualFold(scheme, "http")
case "443":
return strings.EqualFold(scheme, "https")
default:
return false
}
}
// getScope generate a string of a specific date, an AWS region, and a service.
func getScope(t time.Time, region string, service string) string {
scope := strings.Join([]string{
t.Format(yyyymmdd),
region,
service,
"aws4_request",
}, "/")
return scope
}
// getCanonicalRequest generate a canonical request of style
//
// canonicalRequest =
//
// <HTTPMethod>\n
// <CanonicalURI>\n
// <CanonicalQueryString>\n
// <CanonicalHeaders>\n
// <SignedHeaders>\n
// <HashedPayload>
func getCanonicalRequest(extractedSignedHeaders http.Header, payload, queryStr, urlPath, method string) string {
rawQuery := strings.Replace(queryStr, "+", "%20", -1)
encodedPath := encodePath(urlPath)
canonicalRequest := strings.Join([]string{
method,
encodedPath,
rawQuery,
getCanonicalHeaders(extractedSignedHeaders),
getSignedHeaders(extractedSignedHeaders),
payload,
}, "\n")
return canonicalRequest
}
// getStringToSign a string based on selected query values.
func getStringToSign(canonicalRequest string, t time.Time, scope string) string {
stringToSign := signV4Algorithm + "\n" + t.Format(iso8601Format) + "\n"
stringToSign = stringToSign + scope + "\n"
stringToSign = stringToSign + getSHA256Hash([]byte(canonicalRequest))
return stringToSign
}
// getSHA256Hash returns hex-encoded SHA256 hash of the input data.
func getSHA256Hash(data []byte) string {
hash := sha256.Sum256(data)
return hex.EncodeToString(hash[:])
}
// sumHMAC calculate hmac between two input byte array.
func sumHMAC(key []byte, data []byte) []byte {
hash := hmac.New(sha256.New, key)
hash.Write(data)
return hash.Sum(nil)
}
// getSigningKey hmac seed to calculate final signature.
func getSigningKey(secretKey string, time string, region string, service string) []byte {
date := sumHMAC([]byte("AWS4"+secretKey), []byte(time))
regionBytes := sumHMAC(date, []byte(region))
serviceBytes := sumHMAC(regionBytes, []byte(service))
signingKey := sumHMAC(serviceBytes, []byte("aws4_request"))
return signingKey
}
// getCanonicalHeaders generate a list of request headers with their values
func getCanonicalHeaders(signedHeaders http.Header) string {
var headers []string
vals := make(http.Header)
for k, vv := range signedHeaders {
vals[strings.ToLower(k)] = vv
}
for k := range vals {
headers = append(headers, k)
}
sort.Strings(headers)
var buf bytes.Buffer
for _, k := range headers {
buf.WriteString(k)
buf.WriteByte(':')
for idx, v := range vals[k] {
if idx > 0 {
buf.WriteByte(',')
}
buf.WriteString(signV4TrimAll(v))
}
buf.WriteByte('\n')
}
return buf.String()
}
// signV4TrimAll trims leading and trailing spaces from each string in the slice, and trims sequential spaces.
func signV4TrimAll(input string) string {
// Compress adjacent spaces (a space is determined by
// unicode.IsSpace() internally here) to a single space and trim
// leading and trailing spaces.
return strings.Join(strings.Fields(input), " ")
}
// getSignedHeaders generate a string i.e alphabetically sorted, semicolon-separated list of lowercase request header names
func getSignedHeaders(signedHeaders http.Header) string {
var headers []string
for k := range signedHeaders {
headers = append(headers, strings.ToLower(k))
}
sort.Strings(headers)
return strings.Join(headers, ";")
}
// if object matches reserved string, no need to encode them
var reservedObjectNames = regexp.MustCompile("^[a-zA-Z0-9-_.~/]+$")
// encodePath encodes the strings from UTF-8 byte representations to HTML hex escape sequences
//
// This is necessary since regular url.Parse() and url.Encode() functions do not support UTF-8
// non english characters cannot be parsed due to the nature in which url.Encode() is written
//
// This function on the other hand is a direct replacement for url.Encode() technique to support
// pretty much every UTF-8 character.
func encodePath(pathName string) string {
if reservedObjectNames.MatchString(pathName) {
return pathName
}
var encodedPathname string
for _, s := range pathName {
if 'A' <= s && s <= 'Z' || 'a' <= s && s <= 'z' || '0' <= s && s <= '9' { // §2.3 Unreserved characters (mark)
encodedPathname = encodedPathname + string(s)
} else {
switch s {
case '-', '_', '.', '~', '/': // §2.3 Unreserved characters (mark)
encodedPathname = encodedPathname + string(s)
default:
runeLen := utf8.RuneLen(s)
if runeLen < 0 {
return pathName
}
u := make([]byte, runeLen)
utf8.EncodeRune(u, s)
for _, r := range u {
hex := hex.EncodeToString([]byte{r})
encodedPathname = encodedPathname + "%" + strings.ToUpper(hex)
}
}
}
}
return encodedPathname
}
// getSignature final signature in hexadecimal form.
func getSignature(signingKey []byte, stringToSign string) string {
return hex.EncodeToString(sumHMAC(signingKey, []byte(stringToSign)))
}
// compareSignatureV4 returns true if and only if both signatures
// are equal. The signatures are expected to be hex-encoded strings
// according to the AWS S3 signature V4 spec.
func compareSignatureV4(sig1, sig2 string) bool {
// The CTC using []byte(str) works because the hex encoding doesn't use
// non-ASCII characters. Otherwise, we'd need to convert the strings to
// a []rune of UTF-8 characters.
return subtle.ConstantTimeCompare([]byte(sig1), []byte(sig2)) == 1
}