54dd4f091d
* s3api: extend lifecycle XML types with NoncurrentVersionExpiration, AbortIncompleteMultipartUpload Add missing S3 lifecycle rule types to the XML data model: - NoncurrentVersionExpiration with NoncurrentDays and NewerNoncurrentVersions - NoncurrentVersionTransition with NoncurrentDays and StorageClass - AbortIncompleteMultipartUpload with DaysAfterInitiation - Filter.ObjectSizeGreaterThan and ObjectSizeLessThan - And.ObjectSizeGreaterThan and ObjectSizeLessThan - Filter.UnmarshalXML to properly parse Tag, And, and size filter elements Each new type follows the existing set-field pattern for conditional XML marshaling. No behavior changes - these types are not yet wired into handlers or the lifecycle worker. * s3lifecycle: add lifecycle rule evaluator package New package weed/s3api/s3lifecycle/ provides a pure-function lifecycle rule evaluation engine. The evaluator accepts flattened Rule structs and ObjectInfo metadata, and returns the appropriate Action. Components: - evaluator.go: Evaluate() for per-object actions with S3 priority ordering (delete marker > noncurrent version > current expiration), ShouldExpireNoncurrentVersion() with NewerNoncurrentVersions support, EvaluateMPUAbort() for multipart upload rules - filter.go: prefix, tag, and size-based filter matching - tags.go: ExtractTags() extracts S3 tags from filer Extended metadata, HasTagRules() for scan-time optimization - version_time.go: GetVersionTimestamp() extracts timestamps from SeaweedFS version IDs (both old and new format) Comprehensive test coverage: 54 tests covering all action types, filter combinations, edge cases, and version ID formats. * s3api: add UnmarshalXML for Expiration, Transition, ExpireDeleteMarker Add UnmarshalXML methods that set the internal 'set' flag during XML parsing. Previously these flags were only set programmatically, causing XML round-trip to drop elements. This ensures lifecycle configurations stored as XML survive unmarshal/marshal cycles correctly. Add comprehensive XML round-trip tests for all lifecycle rule types including NoncurrentVersionExpiration, AbortIncompleteMultipartUpload, Filter with Tag/And/size constraints, and a complete Terraform-style lifecycle configuration. * s3lifecycle: address review feedback - Fix version_time.go overflow: guard timestampPart > MaxInt64 before the inversion subtraction to prevent uint64 wrap - Make all expiry checks inclusive (!now.Before instead of now.After) so actions trigger at the exact scheduled instant - Add NoncurrentIndex to ObjectInfo so Evaluate() can properly handle NewerNoncurrentVersions via ShouldExpireNoncurrentVersion() - Add test for high-bit overflow version ID * s3lifecycle: guard ShouldExpireNoncurrentVersion against zero SuccessorModTime Add early return when obj.IsLatest or obj.SuccessorModTime.IsZero() to prevent premature expiration of versions with uninitialized successor timestamps (zero value would compute to epoch, always expired). --------- Co-authored-by: Copilot <copilot@github.com>
96 lines
3.0 KiB
Go
96 lines
3.0 KiB
Go
package s3lifecycle
|
|
|
|
import "time"
|
|
|
|
// Rule is a flattened, evaluator-friendly representation of an S3 lifecycle rule.
|
|
// Callers convert from the XML-parsed s3api.Rule (which has nested structs with
|
|
// set-flags for conditional XML marshaling) to this type.
|
|
type Rule struct {
|
|
ID string
|
|
Status string // "Enabled" or "Disabled"
|
|
|
|
// Prefix filter (from Rule.Prefix or Rule.Filter.Prefix or Rule.Filter.And.Prefix).
|
|
Prefix string
|
|
|
|
// Expiration for current versions.
|
|
ExpirationDays int
|
|
ExpirationDate time.Time
|
|
ExpiredObjectDeleteMarker bool
|
|
|
|
// Expiration for non-current versions.
|
|
NoncurrentVersionExpirationDays int
|
|
NewerNoncurrentVersions int
|
|
|
|
// Abort incomplete multipart uploads.
|
|
AbortMPUDaysAfterInitiation int
|
|
|
|
// Tag filter (from Rule.Filter.Tag or Rule.Filter.And.Tags).
|
|
FilterTags map[string]string
|
|
|
|
// Size filters.
|
|
FilterSizeGreaterThan int64
|
|
FilterSizeLessThan int64
|
|
}
|
|
|
|
// ObjectInfo is the metadata about an object that the evaluator uses to
|
|
// determine which lifecycle action applies. Callers build this from filer
|
|
// entry attributes and extended metadata.
|
|
type ObjectInfo struct {
|
|
// Key is the object key relative to the bucket root.
|
|
Key string
|
|
|
|
// ModTime is the object's modification time (entry.Attributes.Mtime).
|
|
ModTime time.Time
|
|
|
|
// Size is the object size in bytes (entry.Attributes.FileSize).
|
|
Size int64
|
|
|
|
// IsLatest is true if this is the current version of the object.
|
|
IsLatest bool
|
|
|
|
// IsDeleteMarker is true if this entry is an S3 delete marker.
|
|
IsDeleteMarker bool
|
|
|
|
// NumVersions is the total number of versions for this object key,
|
|
// including delete markers. Used for ExpiredObjectDeleteMarker evaluation.
|
|
NumVersions int
|
|
|
|
// SuccessorModTime is the creation time of the version that replaced
|
|
// this one (making it non-current). Derived from the successor's version
|
|
// ID timestamp. Zero value for the latest version.
|
|
SuccessorModTime time.Time
|
|
|
|
// NoncurrentIndex is the 0-based position among non-current versions
|
|
// sorted newest-first (0 = newest non-current version). Used by
|
|
// NewerNoncurrentVersions evaluation. -1 or unset for current versions.
|
|
NoncurrentIndex int
|
|
|
|
// Tags are the object's user-defined tags, extracted from the entry's
|
|
// Extended metadata (keys prefixed with "X-Amz-Tagging-").
|
|
Tags map[string]string
|
|
}
|
|
|
|
// Action represents the lifecycle action to take on an object.
|
|
type Action int
|
|
|
|
const (
|
|
// ActionNone means no lifecycle rule applies.
|
|
ActionNone Action = iota
|
|
// ActionDeleteObject deletes the current version of the object.
|
|
ActionDeleteObject
|
|
// ActionDeleteVersion deletes a specific non-current version.
|
|
ActionDeleteVersion
|
|
// ActionExpireDeleteMarker removes a delete marker that is the sole remaining version.
|
|
ActionExpireDeleteMarker
|
|
// ActionAbortMultipartUpload aborts an incomplete multipart upload.
|
|
ActionAbortMultipartUpload
|
|
)
|
|
|
|
// EvalResult is the output of lifecycle rule evaluation.
|
|
type EvalResult struct {
|
|
// Action is the lifecycle action to take.
|
|
Action Action
|
|
// RuleID is the ID of the rule that triggered this action.
|
|
RuleID string
|
|
}
|