* S3: map canned ACL to file permissions and add configurable default file mode
S3 uploads were hardcoded to 0660 regardless of ACL headers. Now the
X-Amz-Acl header maps to Unix file permissions per-object:
- public-read, authenticated-read, bucket-owner-read → 0644
- public-read-write → 0666
- private, bucket-owner-full-control → 0660
Also adds -defaultFileMode / -s3.defaultFileMode flag to set a
server-wide default when no ACL header is present.
Closes#8874
* Address review feedback for S3 file mode feature
- Extract hardcoded 0660 to defaultFileMode constant
- Change parseDefaultFileMode to return error instead of calling Fatalf
- Add -s3.defaultFileMode flag to filer.go and mini.go (was missing)
- Add doc comment to S3Options about updating all four flag sites
- Add TestResolveFileMode with 10 test cases covering ACL mapping,
server default, and priority ordering
* filer: add FilerError enum and error_code field to CreateEntryResponse
Add a machine-readable error code alongside the existing string error
field. This follows the precedent set by PublishMessageResponse in the
MQ broker proto. The string field is kept for human readability and
backward compatibility.
Defined codes: OK, ENTRY_NAME_TOO_LONG, PARENT_IS_FILE,
EXISTING_IS_DIRECTORY, EXISTING_IS_FILE, ENTRY_ALREADY_EXISTS.
* filer: add sentinel errors and error code mapping in filer_pb
Define sentinel errors (ErrEntryNameTooLong, ErrParentIsFile, etc.) in
the filer_pb package so both the filer and consumers can reference them
without circular imports.
Add FilerErrorToSentinel() to map proto error codes to sentinels, and
update CreateEntryWithResponse() to check error_code first, falling back
to the string-based path for backward compatibility with old servers.
* filer: return wrapped sentinel errors and set proto error codes
Replace fmt.Errorf string errors in filer.CreateEntry, UpdateEntry, and
ensureParentDirectoryEntry with wrapped filer_pb sentinel errors (using
%w). This preserves errors.Is() traversal on the server side.
In the gRPC CreateEntry handler, map sentinel errors to the
corresponding FilerError proto codes using errors.Is(), setting both
resp.Error (string, for backward compat) and resp.ErrorCode (enum).
* S3: use errors.Is() with filer sentinels instead of string matching
Replace fragile string-based error matching in filerErrorToS3Error and
other S3 API consumers with errors.Is() checks against filer_pb sentinel
errors. This works because the updated CreateEntryWithResponse helper
reconstructs sentinel errors from the proto FilerError code.
Update iceberg stage_create and metadata_files to check resp.ErrorCode
instead of parsing resp.Error strings. Update SSE-S3 to use errors.Is()
for the already-exists check.
String matching is retained only for non-filer errors (gRPC transport
errors, checksum validation) that don't go through CreateEntryResponse.
* filer: remove backward-compat string fallbacks for error codes
Clients and servers are always deployed together, so there is no need
for backward-compatibility fallback paths that parse resp.Error strings
when resp.ErrorCode is unset. Simplify all consumers to rely solely on
the structured error code.
* iceberg: ensure unknown non-OK error codes are not silently ignored
When FilerErrorToSentinel returns nil for an unrecognized error code,
return an error including the code and message rather than falling
through to return nil.
* filer: fix redundant error message and restore error wrapping in helper
Use request path instead of resp.Error in the sentinel error format
string to avoid duplicating the sentinel message (e.g. "entry already
exists: entry already exists"). Restore %w wrapping with errors.New()
in the fallback paths so callers can use errors.Is()/errors.As().
* filer: promote file to directory on path conflict instead of erroring
S3 allows both "foo/bar" (object) and "foo/bar/xyzzy" (another object)
to coexist because S3 has a flat key space. When ensureParentDirectoryEntry
finds a parent path that is a file instead of a directory, promote it to
a directory by setting ModeDir while preserving the original content and
chunks. Use Store.UpdateEntry directly to bypass the Filer.UpdateEntry
type-change guard.
This fixes the S3 compatibility test failures where creating overlapping
keys (e.g. "foo/bar" then "foo/bar/xyzzy") returned ExistingObjectIsFile.
* S3: add KeyTooLongError error code
Add ErrKeyTooLongError (HTTP 400, code "KeyTooLongError") to match the
standard AWS S3 error for object keys that exceed length limits.
* S3: fix silent PutObject failure when entry name exceeds max_file_name_length
putToFiler called client.CreateEntry() directly and discarded the gRPC
response. The filer embeds application errors like "entry name too long"
in resp.Error (not as gRPC transport errors), so the error was silently
swallowed and clients received HTTP 200 with an ETag for objects that
were never stored.
Switch to the filer_pb.CreateEntry() helper which properly checks
resp.Error, and map "entry name too long" to KeyTooLongError (HTTP 400).
To avoid fragile string parsing across the gRPC boundary, define shared
error message constants in weed/util/constants and use them in both the
filer (producing errors) and S3 API (matching errors). Switch
filerErrorToS3Error to use strings.Contains/HasSuffix with these
constants so matches work regardless of any wrapper prefix. Apply
filerErrorToS3Error to the mkdir path for directory markers.
Fixes#8759
* S3: enforce 1024-byte maximum object key length
AWS S3 limits object keys to 1024 bytes. Add early validation on write
paths (PutObject, CopyObject, CreateMultipartUpload) to reject keys
exceeding the limit with the standard KeyTooLongError (HTTP 400).
The key length check runs before bucket auto-creation to prevent
overlong keys from triggering unnecessary side effects.
Also use filerErrorToS3Error for CopyObject's mkFile error paths so
name-too-long errors from the filer return KeyTooLongError instead of
InternalError.
Ref #8758
* S3: add handler-level tests for key length validation and error mapping
Add tests for filerErrorToS3Error mapping "entry name too long" to
KeyTooLongError, including a regression test for the CreateEntry-prefixed
"existing ... is a directory" form. Add handler-level integration tests
that exercise PutObjectHandler, CopyObjectHandler, and
NewMultipartUploadHandler via httptest, verifying HTTP 400 and
KeyTooLongError XML response for overlong keys and acceptance of keys at
the 1024-byte limit.
When a client cancels an HTTP request (e.g., connection timeout, client
disconnect), the context gets canceled and propagates through the system
as "context canceled" or "code = Canceled" errors. These errors were
being treated as internal server errors (500) when they should be treated
as client errors (400).
Problem:
- Client cancels request or connection times out
- Filer fails to assign file ID with "context canceled"
- S3 API returns HTTP 500 Internal Server Error
- This is incorrect - it's a client issue, not a server issue
Solution:
Added detection for context canceled errors in filerErrorToS3Error():
- Detects "context canceled" and "code = Canceled" in error strings
- Returns ErrInvalidRequest (HTTP 400) instead of ErrInternalError (500)
- Properly attributes the error to the client, not the server
Changes:
- Updated filerErrorToS3Error() to detect context cancellation
- Added test cases for both gRPC and simple context canceled errors
- Maintains existing error handling for other error types
This ensures:
- Clients get appropriate 4xx error codes for their canceled requests
- Server metrics correctly reflect that these are client issues
- Monitoring/alerting won't trigger false positives for client timeouts
Fixes#7060
When an S3 upload has a mismatched Content-MD5 header, SeaweedFS was
incorrectly returning a 500 Internal Server Error instead of the proper
400 Bad Request with error code BadDigest (per AWS S3 specification).
Changes:
- Created weed/util/constants/filer.go with error message constants
- Added ErrMsgBadDigest constant for MD5 mismatch errors
- Added ErrMsgOperationNotPermitted constant for WORM permission errors
- Added ErrBadDigest error code with proper 400 status code mapping
- Updated filerErrorToS3Error() to detect MD5 mismatch and return ErrBadDigest
- Updated filer autoChunk() to return 400 Bad Request for MD5 mismatch
- Refactored error handling to use switch statement for better readability
- Ordered error checks with exact matches first for better maintainability
- Updated all error handling to use centralized constants
- Added comprehensive unit tests
All error messages now use constants from a single location for better
maintainability and consistency. Constants placed in util package to avoid
architectural dependency issues.
Fixes#7305
