Files
seaweedFS/.github/workflows/kms-tests.yml
Chris Lu 17028fbf59 fix: serialize SSE-KMS metadata when bucket default encryption applies KMS (#8780)
* fix: serialize SSE-KMS metadata when bucket default encryption applies KMS

When a bucket has default SSE-KMS encryption enabled and a file is uploaded
without explicit SSE headers, the encryption was applied correctly but the
SSE-KMS metadata (x-seaweedfs-sse-kms-key) was not serialized. This caused
downloads to fail with "empty SSE-KMS metadata" because the entry's Extended
map stored an empty byte slice.

The existing code already handled this for SSE-S3 bucket defaults
(SerializeSSES3Metadata) but was missing the equivalent call to
SerializeSSEKMSMetadata for the KMS path.

Fixes seaweedfs/seaweedfs#8776

* ci: add KMS integration tests to GitHub Actions

Add a kms-tests.yml workflow that runs on changes to KMS/SSE code with
two jobs:

1. KMS provider tests: starts OpenBao via Docker, runs Go integration
   tests in test/kms/ against a real KMS backend
2. S3 KMS e2e tests: starts OpenBao + weed mini built from source, runs
   test_s3_kms.sh which covers bucket-default SSE-KMS upload/download
   (the exact scenario from #8776)

Supporting changes:
- test/kms/Makefile: add CI targets (test-provider-ci, test-s3-kms-ci)
  that manage OpenBao via plain Docker and run weed from source
- test/kms/s3-config-openbao-template.json: S3 config template with
  OpenBao KMS provider for weed mini

* refactor: combine SSE-S3 and SSE-KMS metadata serialization into else-if

SSE-S3 and SSE-KMS bucket default encryption are mutually exclusive, so
use a single if/else-if block instead of two independent if blocks.

* Update .github/workflows/kms-tests.yml

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

* fix(ci): start weed mini from data dir to avoid Docker filer.toml

weed mini reads filer.toml from the current working directory first.
When running from test/kms/, it picked up the Docker-targeted filer.toml
which has dir="/data/filerdb" (a path that doesn't exist in CI), causing
a fatal crash at filer store initialization.

Fix by cd-ing to the data directory before starting weed mini.
Also improve log visibility on failure.

---------

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
2026-03-26 14:07:01 -07:00

141 lines
3.6 KiB
YAML

name: "KMS Tests"
on:
pull_request:
paths:
- 'weed/kms/**'
- 'weed/s3api/s3_sse_*.go'
- 'weed/s3api/s3api_object_handlers.go'
- 'weed/s3api/s3api_object_handlers_put.go'
- 'test/kms/**'
- '.github/workflows/kms-tests.yml'
push:
branches: [ master, main ]
paths:
- 'weed/kms/**'
- 'weed/s3api/s3_sse_*.go'
- 'weed/s3api/s3api_object_handlers.go'
- 'weed/s3api/s3api_object_handlers_put.go'
- 'test/kms/**'
concurrency:
group: ${{ github.head_ref || github.ref }}-kms-tests
cancel-in-progress: true
permissions:
contents: read
defaults:
run:
working-directory: weed
jobs:
kms-provider-tests:
name: KMS Provider Integration Tests
runs-on: ubuntu-22.04
timeout-minutes: 20
steps:
- name: Check out code
uses: actions/checkout@v6
- name: Set up Go
uses: actions/setup-go@v6
with:
go-version-file: 'go.mod'
id: go
- name: Install SeaweedFS
run: |
go install -buildvcs=false
- name: Run KMS provider integration tests
timeout-minutes: 15
working-directory: test/kms
run: |
set -x
echo "=== System Information ==="
uname -a
free -h
docker --version
make test-provider-ci
- name: Show OpenBao logs on failure
if: failure()
run: |
echo "=== OpenBao Container Logs ==="
docker logs openbao-ci 2>&1 | tail -50 || echo "No OpenBao container found"
echo "=== Setup Logs ==="
cat /tmp/openbao-ci-setup.log 2>/dev/null || echo "No setup log found"
- name: Cleanup
if: always()
working-directory: test/kms
run: |
make stop-openbao-ci || true
s3-kms-e2e-tests:
name: S3 KMS End-to-End Tests
runs-on: ubuntu-22.04
timeout-minutes: 25
steps:
- name: Check out code
uses: actions/checkout@v6
- name: Set up Go
uses: actions/setup-go@v6
with:
go-version-file: 'go.mod'
id: go
- name: Install SeaweedFS
run: |
go install -buildvcs=false
- name: Run S3 KMS end-to-end tests
timeout-minutes: 20
working-directory: test/kms
run: |
set -x
echo "=== System Information ==="
uname -a
free -h
docker --version
aws --version
make test-s3-kms-ci
- name: Show logs on failure
if: failure()
working-directory: test/kms
run: |
echo "=== OpenBao Container Logs ==="
cat /tmp/openbao-ci-container.log 2>/dev/null || docker logs openbao-ci 2>&1 | tail -50 || echo "No OpenBao logs found"
echo "=== SeaweedFS Server Logs ==="
tail -100 /tmp/seaweedfs-kms-mini.log 2>/dev/null || echo "No server log found"
echo "=== Setup Logs ==="
cat /tmp/weed-kms-ci-setup.log 2>/dev/null || echo "No weed setup log"
echo "=== Process Information ==="
ps aux | grep -E "(weed|test)" || true
- name: Upload test logs on failure
if: failure()
uses: actions/upload-artifact@v7
with:
name: s3-kms-e2e-logs
path: |
/tmp/seaweedfs-kms-mini.log
/tmp/openbao-ci-container.log
/tmp/weed-kms-ci-setup.log
retention-days: 3
- name: Cleanup
if: always()
working-directory: test/kms
run: |
make stop-seaweedfs-ci || true
make stop-openbao-ci || true
make clean-ci || true