More efficient copy object (#6665)
* it compiles * refactored * reduce to 4 concurrent chunk upload * CopyObjectPartHandler * copy a range of the chunk data, fix offset size in copied chunks * Update s3api_object_handlers_copy.go What the PR Accomplishes: CopyObjectHandler - Now copies entire objects by copying chunks individually instead of downloading/uploading the entire file CopyObjectPartHandler - Handles copying parts of objects for multipart uploads by copying only the relevant chunk portions Efficient Chunk Copying - Uses direct chunk-to-chunk copying with proper volume assignment and concurrent processing (limited to 4 concurrent operations) Range Support - Properly handles range-based copying for partial object copies * fix compilation * fix part destination * handling small objects * use mkFile * copy to existing file or part * add testing tools * adjust tests * fix chunk lookup * refactoring * fix TestObjectCopyRetainingMetadata * ensure bucket name not conflicting * fix conditional copying tests * remove debug messages * add custom s3 copy tests
This commit is contained in:
@@ -93,7 +93,7 @@ func (s3a *S3ApiServer) completeMultipartUpload(input *s3.CompleteMultipartUploa
|
||||
completedPartNumbers = append(completedPartNumbers, part.PartNumber)
|
||||
}
|
||||
completedPartMap[part.PartNumber] = append(completedPartMap[part.PartNumber], part.ETag)
|
||||
maxPartNo = max(maxPartNo, part.PartNumber)
|
||||
maxPartNo = maxInt(maxPartNo, part.PartNumber)
|
||||
}
|
||||
sort.Ints(completedPartNumbers)
|
||||
|
||||
@@ -444,3 +444,11 @@ func (s3a *S3ApiServer) listObjectParts(input *s3.ListPartsInput) (output *ListP
|
||||
|
||||
return
|
||||
}
|
||||
|
||||
// maxInt returns the maximum of two int values
|
||||
func maxInt(a, b int) int {
|
||||
if a > b {
|
||||
return a
|
||||
}
|
||||
return b
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user