gsh-digital-services/seaweedFS
5
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix: serialize SSE-KMS metadata when bucket default encryption applies KMS (#8780)

Browse Source 
* fix: serialize SSE-KMS metadata when bucket default encryption applies KMS

When a bucket has default SSE-KMS encryption enabled and a file is uploaded
without explicit SSE headers, the encryption was applied correctly but the
SSE-KMS metadata (x-seaweedfs-sse-kms-key) was not serialized. This caused
downloads to fail with "empty SSE-KMS metadata" because the entry's Extended
map stored an empty byte slice.

The existing code already handled this for SSE-S3 bucket defaults
(SerializeSSES3Metadata) but was missing the equivalent call to
SerializeSSEKMSMetadata for the KMS path.

Fixes seaweedfs/seaweedfs#8776

* ci: add KMS integration tests to GitHub Actions

Add a kms-tests.yml workflow that runs on changes to KMS/SSE code with
two jobs:

1. KMS provider tests: starts OpenBao via Docker, runs Go integration
   tests in test/kms/ against a real KMS backend
2. S3 KMS e2e tests: starts OpenBao + weed mini built from source, runs
   test_s3_kms.sh which covers bucket-default SSE-KMS upload/download
   (the exact scenario from #8776)

Supporting changes:
- test/kms/Makefile: add CI targets (test-provider-ci, test-s3-kms-ci)
  that manage OpenBao via plain Docker and run weed from source
- test/kms/s3-config-openbao-template.json: S3 config template with
  OpenBao KMS provider for weed mini

* refactor: combine SSE-S3 and SSE-KMS metadata serialization into else-if

SSE-S3 and SSE-KMS bucket default encryption are mutually exclusive, so
use a single if/else-if block instead of two independent if blocks.

* Update .github/workflows/kms-tests.yml

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

* fix(ci): start weed mini from data dir to avoid Docker filer.toml

weed mini reads filer.toml from the current working directory first.
When running from test/kms/, it picked up the Docker-targeted filer.toml
which has dir="/data/filerdb" (a path that doesn't exist in CI), causing
a fatal crash at filer store initialization.

Fix by cd-ing to the data directory before starting weed mini.
Also improve log visibility on failure.

---------

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
This commit is contained in:
Chris Lu
2026-03-26 14:07:01 -07:00
committed by GitHub
parent 3a3fff1399
commit 17028fbf59
4 changed files with 320 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files

10
weed/s3api/s3api_object_handlers_put.go
View File

@@ -365,14 +365,20 @@ func (s3a *S3ApiServer) putToFiler(r *http.Request, filePath string, dataReader
}
}
// If SSE-S3 was applied by bucket default, prepare metadata (if not already done)
// If bucket default encryption was applied, serialize the metadata (SSE-S3 and SSE-KMS are mutually exclusive)
var metaErr error
if sseS3Key != nil && len(sseS3Metadata) == 0 {
var metaErr error
sseS3Metadata, metaErr = SerializeSSES3Metadata(sseS3Key)
if metaErr != nil {
glog.Errorf("Failed to serialize SSE-S3 metadata for bucket default encryption: %v", metaErr)
return "", s3err.ErrInternalError, SSEResponseMetadata{}
}
} else if sseKMSKey != nil && len(sseKMSMetadata) == 0 {
sseKMSMetadata, metaErr = SerializeSSEKMSMetadata(sseKMSKey)
if metaErr != nil {
glog.Errorf("Failed to serialize SSE-KMS metadata for bucket default encryption: %v", metaErr)
return "", s3err.ErrInternalError, SSEResponseMetadata{}
}
}
} else {
glog.V(4).Infof("putToFiler: explicit encryption already applied, skipping bucket default encryption")