8 Commits

Author SHA1 Message Date
Ping Qiu
5c8de5e282 fix: close volumes and EC shards in tests for Windows compatibility (#8152)
* fix: close volumes and EC shards in tests to prevent Windows cleanup failures

On Windows, t.TempDir() cleanup fails when test files are still open
because Windows enforces mandatory file locking. Add defer v.Close(),
defer store.Close(), and EC volume cleanup to ensure all file handles
are released before temp directory removal.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: extract closeEcVolumes helper to reduce duplication

Address code review feedback by extracting the repeated EC volume
cleanup loop into a closeEcVolumes() helper function.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-28 17:21:14 -08:00
Chris Lu
ade2e58cf4 refactoring 2026-01-01 19:20:59 -08:00
Chris Lu
0813138d57 Volume Server: handle incomplete ec encoding (#7384)
* handle incomplete ec encoding

* unit tests

* simplify, and better logs

* Update disk_location_ec.go

When loadEcShards() fails partway through, some EC shards may already be loaded into the l.ecVolumes map in memory. The previous code only cleaned up filesystem files but left orphaned in-memory state, which could cause memory leaks and inconsistent state.

* address comments

* Performance: Avoid Double os.Stat() Call
* Platform Compatibility: Use filepath.Join

* in memory cleanup

* Update disk_location_ec.go

* refactor

* Added Shard Size Validation

* check ec shard sizes

* validate shard size

* calculate expected shard size

* refactoring

* minor

* fix shard directory

* 10GB sparse files can be slow or fail on non-sparse FS. Use 10MB to hit SmallBlockSize math (1MB shards) deterministically.

* grouping logic should be updated to use both collection and volumeId to ensure correctness

* unexpected error

* handle exceptions in tests; use constants

* The check for orphaned shards should be performed for the previous volume before resetting sameVolumeShards for the new volume.

* address comments

* Eliminated Redundant Parsing in checkOrphanedShards

* minor

* Avoid misclassifying local EC as distributed when .dat stat errors occur; also standardize unload-before-remove.

* fmt

* refactor

* refactor

* adjust to warning
2025-10-26 22:48:58 -07:00
Chris Lu
0e52862586 avoid the test that depends on ordering in a different folder 2019-06-04 02:22:32 -07:00
Chris Lu
2f7710a75d debug sporadic test error on travis 2019-06-04 02:09:06 -07:00
Chris Lu
3f9ecee40f working with reading remote intervals 2019-05-28 21:29:07 -07:00
Chris Lu
217cde0a3b refactoring 2019-05-27 21:40:51 -07:00
Chris Lu
17ac1290c0 volume: load ec shards during heartbeats to master 2019-05-21 22:41:20 -07:00