ec: fall back to data dir when ecx file not found in idx dir (#8541)

* ec: fall back to data dir when ecx file not found in idx dir (#8540)

When -dir.idx is configured after EC encoding, the .ecx/.ecj files
remain in the data directory. NewEcVolume now falls back to the data
directory when the index file is not found in dirIdx.

* ec: add fallback logging and improved error message for ecx lookup

* ec: preserve configured dirIdx, track actual ecx location separately

The previous fallback set ev.dirIdx = dir when finding .ecx in the data
directory, which corrupted IndexBaseFileName() for future writes (e.g.,
WriteIdxFileFromEcIndex during EC-to-volume conversion would write the
.idx file to the data directory instead of the configured index directory).

Introduce ecxActualDir to track where .ecx/.ecj were actually found,
used only by FileName() for cleanup/destroy. IndexBaseFileName() continues
to use the configured dirIdx for new file creation.

* ec: check both idx and data dirs for .ecx in all cleanup and lookup paths

When -dir.idx is configured after EC encoding, .ecx/.ecj files may
reside in the data directory. Several code paths only checked
l.IdxDirectory, causing them to miss these files:

- removeEcVolumeFiles: now removes .ecx/.ecj from both directories
- loadExistingVolume: ecx existence check falls back to data dir
- deleteEcShardIdsForEachLocation: ecx existence check and cleanup
  both cover the data directory
- VolumeEcShardsRebuild: ecx lookup falls back to data directory
  so RebuildEcxFile operates on the correct file
This commit is contained in:
Chris Lu
2026-03-07 09:18:48 -08:00
committed by GitHub
parent bfc430afbd
commit af4c3fcb31
4 changed files with 43 additions and 7 deletions

View File

@@ -28,6 +28,7 @@ type EcVolume struct {
Collection string
dir string
dirIdx string
ecxActualDir string // directory where .ecx/.ecj were actually found (may differ from dirIdx after fallback)
ecxFile *os.File
ecxFileSize int64
ecxCreatedAt time.Time
@@ -51,8 +52,20 @@ func NewEcVolume(diskType types.DiskType, dir string, dirIdx string, collection
indexBaseFileName := EcShardFileName(collection, dirIdx, int(vid))
// open ecx file
ev.ecxActualDir = dirIdx
if ev.ecxFile, err = os.OpenFile(indexBaseFileName+".ecx", os.O_RDWR, 0644); err != nil {
return nil, fmt.Errorf("cannot open ec volume index %s.ecx: %v", indexBaseFileName, err)
if dirIdx != dir && os.IsNotExist(err) {
// fall back to data directory if idx directory does not have the .ecx file
firstErr := err
glog.V(1).Infof("ecx file not found at %s.ecx, falling back to %s.ecx", indexBaseFileName, dataBaseFileName)
if ev.ecxFile, err = os.OpenFile(dataBaseFileName+".ecx", os.O_RDWR, 0644); err != nil {
return nil, fmt.Errorf("open ecx index %s.ecx: %v; fallback %s.ecx: %v", indexBaseFileName, firstErr, dataBaseFileName, err)
}
indexBaseFileName = dataBaseFileName
ev.ecxActualDir = dir
} else {
return nil, fmt.Errorf("cannot open ec volume index %s.ecx: %v", indexBaseFileName, err)
}
}
ecxFi, statErr := ev.ecxFile.Stat()
if statErr != nil {
@@ -197,7 +210,7 @@ func (ev *EcVolume) Destroy() {
func (ev *EcVolume) FileName(ext string) string {
switch ext {
case ".ecx", ".ecj":
return ev.IndexBaseFileName() + ext
return EcShardFileName(ev.Collection, ev.ecxActualDir, int(ev.VolumeId)) + ext
}
// .vif
return ev.DataBaseFileName() + ext