plugin scheduler: run iceberg and lifecycle lanes concurrently (#8821)
* plugin scheduler: run iceberg and lifecycle lanes concurrently The default lane serialises job types under a single admin lock because volume-management operations share global state. Iceberg and lifecycle lanes have no such constraint, so run each of their job types independently in separate goroutines. * Fix concurrent lane scheduler status * plugin scheduler: address review feedback - Extract collectDueJobTypes helper to deduplicate policy loading between locked and concurrent iteration paths. - Use atomic.Bool instead of sync.Mutex for hadJobs in the concurrent path. - Set lane loop state to "busy" before launching concurrent goroutines so the lane is not reported as idle while work runs. - Convert TestLaneRequiresLock to table-driven style. - Add TestRunLaneSchedulerIterationLockBehavior to verify the scheduler acquires the admin lock only for lanes that require it. - Fix flaky TestGetLaneSchedulerStatusShowsActiveConcurrentLaneWork by not starting background scheduler goroutines that race with the direct runJobTypeIteration call.
This commit is contained in:
@@ -38,6 +38,25 @@ var laneIdleSleep = map[SchedulerLane]time.Duration{
|
||||
LaneLifecycle: 5 * time.Minute,
|
||||
}
|
||||
|
||||
// laneRequiresLock maps each lane to whether its job types must be
|
||||
// serialised under a single admin lock. The default lane needs this
|
||||
// because volume-management operations share global state. Other
|
||||
// lanes run each job type independently.
|
||||
var laneRequiresLock = map[SchedulerLane]bool{
|
||||
LaneDefault: true,
|
||||
LaneIceberg: false,
|
||||
LaneLifecycle: false,
|
||||
}
|
||||
|
||||
// LaneRequiresLock returns true if the given lane needs a single admin
|
||||
// lock to serialise its job types. Unknown lanes default to true.
|
||||
func LaneRequiresLock(lane SchedulerLane) bool {
|
||||
if v, ok := laneRequiresLock[lane]; ok {
|
||||
return v
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
// LaneIdleSleep returns the idle sleep duration for the given lane,
|
||||
// falling back to defaultSchedulerIdleSleep if the lane is unknown.
|
||||
func LaneIdleSleep(lane SchedulerLane) time.Duration {
|
||||
|
||||
Reference in New Issue
Block a user