Files
seaweedFS/weed/util
Chris Lu b665c329bc fix(replication): resume partial chunk reads on EOF instead of re-downloading (#8607)
* fix(replication): resume partial chunk reads on EOF instead of re-downloading

When replicating chunks and the source connection drops mid-transfer,
accumulate the bytes already received and retry with a Range header
to fetch only the remaining bytes. This avoids re-downloading
potentially large chunks from scratch on each retry, reducing load
on busy source servers and speeding up recovery.

* test(replication): add tests for downloadWithRange including gzip partial reads

Tests cover:
- No offset (no Range header sent)
- With offset (Range header verified)
- Content-Disposition filename extraction
- Partial read + resume: server drops connection mid-transfer, client
  resumes with Range from the offset of received bytes
- Gzip partial read + resume: first response is gzip-encoded (Go auto-
  decompresses), connection drops, resume request gets decompressed data
  (Go doesn't add Accept-Encoding when Range is set, so the server
  decompresses), combined bytes match original

* fix(replication): address PR review comments

- Consolidate downloadWithRange into DownloadFile with optional offset
  parameter (variadic), eliminating code duplication (DRY)
- Validate HTTP response status: require 206 + correct Content-Range
  when offset > 0, reject when server ignores Range header
- Use if/else for fullData assignment for clarity
- Add test for rejected Range (server returns 200 instead of 206)

* refactor(replication): remove unused ReplicationSource interface

The interface was never referenced and its signature didn't match
the actual FilerSource.ReadPart method.

---------

Co-authored-by: Copilot <copilot@github.com>
2026-03-11 22:38:22 -07:00
..
2023-11-26 11:47:20 -08:00
2026-02-20 18:42:00 -08:00
2019-11-17 11:40:36 +08:00
2026-03-11 02:29:24 -07:00
2021-04-27 10:37:24 +08:00
2021-08-20 18:38:18 +08:00
2025-10-31 12:49:04 -07:00
2026-02-20 18:42:00 -08:00
2026-02-20 18:42:00 -08:00
2026-02-20 18:42:00 -08:00
2026-02-20 18:42:00 -08:00
2026-02-20 18:42:00 -08:00
2026-02-20 18:42:00 -08:00
2022-09-25 17:43:42 -07:00
2022-09-25 13:34:55 -07:00
2023-09-21 23:51:32 -07:00
2021-04-27 10:37:24 +08:00
2024-10-31 08:40:05 -07:00