optimize: enable immediate EC shard reporting during startup (#7933)
* optimize: enable immediate EC shard reporting during startup
Ported the immediate EC shard reporting feature from Enterprise to Community version.
This allows the master to be notified about EC shards immediately during volume server startup,
instead of waiting for the first heartbeat.
Changes:
1. Updated NewStore to initialize notification channels BEFORE loading volumes (fixes potential nil panic).
2. Added ecShardNotifyHandler to report EC shards to NewEcShardsChan during startup.
3. Implemented non-blocking channel send for EC reporting to prevent deadlock when loading many EC shards (fixing the enterprise bug 17ac1290c).
4. Updated DiskLocation and EC loading logic to support the callback.
This optimization improves cluster state consistency and startup speed for EC-heavy clusters.
* optimize: report actual EC shard size during startup
* optimize: increase notification channel buffer size to 1024
* optimize: fix variable shadowing in store.go
This commit is contained in:
@@ -34,6 +34,8 @@ type DiskLocation struct {
|
||||
ecVolumes map[needle.VolumeId]*erasure_coding.EcVolume
|
||||
ecVolumesLock sync.RWMutex
|
||||
|
||||
ecShardNotifyHandler func(collection string, vid needle.VolumeId, shardId erasure_coding.ShardId, ecVolume *erasure_coding.EcVolume)
|
||||
|
||||
isDiskSpaceLow bool
|
||||
closeCh chan struct{}
|
||||
}
|
||||
@@ -259,7 +261,7 @@ func (l *DiskLocation) loadExistingVolumesWithId(needleMapKind NeedleMapKind, ld
|
||||
l.concurrentLoadingVolumes(needleMapKind, workerNum, ldbTimeout, diskId)
|
||||
glog.V(2).Infof("Store started on dir: %s with %d volumes max %d (disk ID: %d)", l.Directory, len(l.volumes), l.MaxVolumeCount, diskId)
|
||||
|
||||
l.loadAllEcShards()
|
||||
l.loadAllEcShardsWithCallback(l.ecShardNotifyHandler)
|
||||
glog.V(2).Infof("Store started on dir: %s with %d ec shards (disk ID: %d)", l.Directory, len(l.ecVolumes), diskId)
|
||||
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user