* Process .ecj deletions during EC decode and vacuum decoded volume (#8798)
When decoding EC volumes back to normal volumes, deletions recorded in
the .ecj journal were not being applied before computing the dat file
size or checking for live needles. This caused the decoded volume to
include data for deleted files and could produce false positives in the
all-deleted check.
- Call RebuildEcxFile before HasLiveNeedles/FindDatFileSize in
VolumeEcShardsToVolume so .ecj deletions are merged into .ecx first
- Vacuum the decoded volume after mounting in ec.decode to compact out
deleted needle data from the .dat file
- Add integration tests for decoding with non-empty .ecj files
* storage: add offline volume compaction helper
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* ec: compact decoded volumes before deleting shards
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* ec: address PR review comments
- Fall back to data directory for .ecx when idx directory lacks it
- Make compaction failure non-fatal during EC decode
- Remove misleading "buffer: 10%" from space check error message
* ec: collect .ecj from all shard locations during decode
Each server's .ecj only contains deletions for needles whose data
resides in shards held by that server. Previously, sources with no
new data shards to contribute were skipped entirely, losing their
.ecj deletion entries. Now .ecj is always appended from every shard
location so RebuildEcxFile sees the full set of deletions.
* ec: add integration tests for .ecj collection during decode
TestEcDecodePreservesDeletedNeedles: verifies that needles deleted
via VolumeEcBlobDelete are excluded from the decoded volume.
TestEcDecodeCollectsEcjFromPeer: regression test for the fix in
collectEcShards. Deletes a needle only on a peer server that holds
no new data shards, then verifies the deletion survives decode via
.ecj collection.
* ec: address review nits in decode and tests
- Remove double error wrapping in mountDecodedVolume
- Check VolumeUnmount error in peer ecj test
- Assert 404 specifically for deleted needles, fail on 5xx
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: sync EC volume files before copying to fix deleted needles not being marked when decoding (#7751)
When a file is deleted from an EC volume, the deletion is written to both
the .ecx and .ecj files. However, these writes were not synced to disk
before the files were copied during ec.decode. This caused the copied
files to miss the deletion markers, resulting in 'leaked' space where
deleted files were not properly tracked after decoding.
This fix:
1. Adds a Sync() method to EcVolume that flushes .ecx and .ecj files
to disk without closing them
2. Calls Sync() in CopyFile before copying EC volume files to ensure
all deletions are visible to the copy operation
Fixes#7751
* test: add integration tests for EC volume deletion sync (issue #7751)
Add comprehensive tests to verify that deleted needles are properly
visible after EcVolume.Sync() is called. These tests cover:
1. TestWriteIdxFileFromEcIndex_PreservesDeletedNeedles
- Verifies that WriteIdxFileFromEcIndex preserves deletion markers
from .ecx files when generating .idx files
2. TestWriteIdxFileFromEcIndex_ProcessesEcjJournal
- Verifies that deletions from .ecj journal file are correctly
appended to the generated .idx file
3. TestEcxFileDeletionVisibleAfterSync
- Verifies that MarkNeedleDeleted changes are visible after Sync()
4. TestEcxFileDeletionWithSeparateHandles
- Tests that synced changes are visible across separate file handles
5. TestEcVolumeSyncEnsuresDeletionsVisible
- Integration test for the full EcVolume.DeleteNeedleFromEcx +
Sync() workflow that validates the fix for issue #7751
* refactor: log sync errors in EcVolume.Sync() instead of ignoring them
Per code review feedback: sync errors could reintroduce the bug this PR
fixes, so logging warnings helps with debugging.
* fix(ec.decode): purge EC shards when volume is empty
When an EC volume has no live entries (all deleted), ec.decode should not generate an empty normal volume. Instead, treat decode as a no-op and allow shard purge to proceed cleanly.\n\nFixes: #7748
* chore: address PR review comments
* test: cover live EC index + avoid magic string
* chore: harden empty-EC handling
- Make shard cleanup best-effort (collect errors)\n- Remove unreachable EOF handling in HasLiveNeedles\n- Add empty ecx test case\n- Share no-live-entries substring between server/client\n
* perf: parallelize EC shard unmount/delete across locations
* refactor: combine unmount+delete into single goroutine per location
* refactor: use errors.Join for multi-error aggregation
* refactor: use existing ErrorWaitGroup for parallel execution
* fix: capture loop variables + clarify SuperBlockSize safety
