s3: fix presigned POST upload missing slash between bucket and key (#7714)

* s3: fix presigned POST upload missing slash between bucket and key

When uploading a file using presigned POST (e.g., boto3.generate_presigned_post),
the file was saved with the bucket name and object key concatenated without a
slash (e.g., 'my-bucketfilename' instead of 'my-bucket/filename').

The issue was that PostPolicyBucketHandler retrieved the object key from form
values without ensuring it had a leading slash, unlike GetBucketAndObject()
which normalizes the key.

Fixes #7713

* s3: add tests for presigned POST key normalization

Add comprehensive tests for PostPolicyBucketHandler to ensure:
- Object keys without leading slashes are properly normalized
- ${filename} substitution works correctly with normalization
- Path construction correctly separates bucket and key
- Form value extraction works properly

These tests would have caught the bug fixed in the previous commit
where keys like 'test_image.png' were concatenated with bucket
without a separator, resulting in 'my-buckettest_image.png'.

* s3: create normalizeObjectKey function for robust key normalization

Address review feedback by creating a reusable normalizeObjectKey function
that both adds a leading slash and removes duplicate slashes, aligning with
how other handlers process paths (e.g., toFilerPath uses removeDuplicateSlashes).

The function handles edge cases like:
- Keys without leading slashes (the original bug)
- Keys with duplicate slashes (e.g., 'a//b' -> '/a/b')
- Keys with leading duplicate slashes (e.g., '///a' -> '/a')

Updated tests to use the new function and added TestNormalizeObjectKey
for comprehensive coverage of the new function.

* s3: move NormalizeObjectKey to s3_constants for shared use

Move the NormalizeObjectKey function to the s3_constants package so it can
be reused by:
- GetBucketAndObject() - now normalizes all object keys from URL paths
- GetPrefix() - now normalizes prefix query parameters
- PostPolicyBucketHandler - normalizes keys from form values

This ensures consistent object key normalization across all S3 API handlers,
handling both missing leading slashes and duplicate slashes.

Benefits:
- Single source of truth for key normalization
- GetBucketAndObject now removes duplicate slashes (previously only added leading slash)
- All handlers benefit from the improved normalization automatically
This commit is contained in:
Chris Lu
2025-12-10 23:42:58 -08:00
committed by GitHub
parent df4f2f7020
commit de3ecaf0de
4 changed files with 414 additions and 3 deletions

View File

@@ -140,17 +140,44 @@ const (
func GetBucketAndObject(r *http.Request) (bucket, object string) {
vars := mux.Vars(r)
bucket = vars["bucket"]
object = vars["object"]
object = NormalizeObjectKey(vars["object"])
return
}
// NormalizeObjectKey ensures the object key has a leading slash and no duplicate slashes.
// This normalizes keys from various sources (URL path, form values, etc.) to a consistent format.
func NormalizeObjectKey(object string) string {
object = removeDuplicateSlashes(object)
if !strings.HasPrefix(object, "/") {
object = "/" + object
}
return object
}
return
// removeDuplicateSlashes removes consecutive slashes from a path
func removeDuplicateSlashes(s string) string {
var result strings.Builder
result.Grow(len(s))
lastWasSlash := false
for _, r := range s {
if r == '/' {
if !lastWasSlash {
result.WriteRune(r)
}
lastWasSlash = true
} else {
result.WriteRune(r)
lastWasSlash = false
}
}
return result.String()
}
func GetPrefix(r *http.Request) string {
query := r.URL.Query()
prefix := query.Get("prefix")
prefix = removeDuplicateSlashes(prefix)
if !strings.HasPrefix(prefix, "/") {
prefix = "/" + prefix
}