Files
seaweedFS/weed
Chris Lu d26c260041 s3: fix memory leak in ListObjectVersions with early termination (#7785)
* s3: fix memory leak in ListObjectVersions with early termination

This fixes a critical memory leak in S3 versioned bucket listing operations:

1. Add maxCollect parameter to findVersionsRecursively() to stop collecting
   versions once we have enough for the response + truncation detection

2. Add early termination checks throughout the recursive traversal to
   prevent scanning entire buckets when only a small number of results
   are requested

3. Use clear() on tracking maps after collection to help GC reclaim memory

4. Create new slice with exact capacity when truncating results instead of
   re-slicing, which allows GC to reclaim the excess backing array memory

5. Pre-allocate result slice with reasonable initial capacity to reduce
   reallocations during collection

Before this fix, listing versions on a bucket with many objects and versions
would load ALL versions into memory before pagination, causing OOM crashes.

Fixes memory exhaustion when calling ListObjectVersions on large versioned buckets.

* s3: fix pre-allocation capacity to be consistent with maxCollect

Address review feedback: the previous capping logic caused an inconsistency
where initialCap was capped to 1000 but maxCollect was maxKeys+1, leading to
unnecessary reallocations when maxKeys was 1000.

Fix by:
1. Cap maxKeys to 1000 (S3 API limit) at the start of the function
2. Use maxKeys+1 directly for slice capacity, ensuring consistency with
   the maxCollect parameter passed to findVersionsRecursively
2025-12-15 22:43:09 -08:00
..
2025-10-13 18:05:17 -07:00
2024-06-25 09:18:11 -07:00
2024-02-14 08:26:38 -08:00
2025-12-15 01:06:55 -08:00
2025-03-17 23:13:27 -07:00