s3: support STREAMING-AWS4-HMAC-SHA256-PAYLOAD-TRAILER for signed chunked uploads with checksums (#7623)
* s3: support STREAMING-AWS4-HMAC-SHA256-PAYLOAD-TRAILER for signed chunked uploads with checksums When AWS SDK v2 clients upload with both chunked encoding and checksum validation enabled, they use the x-amz-content-sha256 header value of STREAMING-AWS4-HMAC-SHA256-PAYLOAD-TRAILER instead of the simpler STREAMING-AWS4-HMAC-SHA256-PAYLOAD. This caused the chunked reader to not be properly activated, resulting in chunk-signature metadata being stored as part of the file content. Changes: - Add streamingSignedPayloadTrailer constant for the new header value - Update isRequestSignStreamingV4() to recognize this header - Update newChunkedReader() to handle this streaming type - Update calculateSeedSignature() to accept this header - Add unit test for signed streaming upload with trailer Fixes issue where Quarkus/AWS SDK v2 uploads with checksum validation resulted in corrupted file content containing chunk-signature data. * address review comments: add trailer signature to test, fix constant alignment * test: separate canonical trailer text (\n) from on-wire format (\r\n) * test: add negative test for invalid trailer signature * refactor: check HTTP method first in streaming auth checks (fail-fast) * test: handle crc32 Write error return for completeness * refactor: extract createTrailerStreamingRequest helper to reduce test duplication * fmt * docs: clarify test comment about trailer signature validation status * refactor: calculate chunk data length dynamically instead of hardcoding * Update weed/s3api/chunked_reader_v4_test.go Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * fix: use current time for signatures instead of hardcoded past date --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
This commit is contained in:
@@ -48,14 +48,22 @@ func isRequestPostPolicySignatureV4(r *http.Request) bool {
|
||||
}
|
||||
|
||||
// Verify if the request has AWS Streaming Signature Version '4'. This is only valid for 'PUT' operation.
|
||||
// Supports both with and without trailer variants:
|
||||
// - STREAMING-AWS4-HMAC-SHA256-PAYLOAD (original)
|
||||
// - STREAMING-AWS4-HMAC-SHA256-PAYLOAD-TRAILER (with trailing checksums)
|
||||
func isRequestSignStreamingV4(r *http.Request) bool {
|
||||
return r.Header.Get("x-amz-content-sha256") == streamingContentSHA256 &&
|
||||
r.Method == http.MethodPut
|
||||
if r.Method != http.MethodPut {
|
||||
return false
|
||||
}
|
||||
contentSha256 := r.Header.Get("x-amz-content-sha256")
|
||||
return contentSha256 == streamingContentSHA256 || contentSha256 == streamingContentSHA256Trailer
|
||||
}
|
||||
|
||||
func isRequestUnsignedStreaming(r *http.Request) bool {
|
||||
return r.Header.Get("x-amz-content-sha256") == streamingUnsignedPayload &&
|
||||
r.Method == http.MethodPut
|
||||
if r.Method != http.MethodPut {
|
||||
return false
|
||||
}
|
||||
return r.Header.Get("x-amz-content-sha256") == streamingUnsignedPayload
|
||||
}
|
||||
|
||||
// Authorization type.
|
||||
|
||||
Reference in New Issue
Block a user