Refactor plugin system and migrate worker runtime (#8369)
* admin: add plugin runtime UI page and route wiring * pb: add plugin gRPC contract and generated bindings * admin/plugin: implement worker registry, runtime, monitoring, and config store * admin/dash: wire plugin runtime and expose plugin workflow APIs * command: add flags to enable plugin runtime * admin: rename remaining plugin v2 wording to plugin * admin/plugin: add detectable job type registry helper * admin/plugin: add scheduled detection and dispatch orchestration * admin/plugin: prefetch job type descriptors when workers connect * admin/plugin: add known job type discovery API and UI * admin/plugin: refresh design doc to match current implementation * admin/plugin: enforce per-worker scheduler concurrency limits * admin/plugin: use descriptor runtime defaults for scheduler policy * admin/ui: auto-load first known plugin job type on page open * admin/plugin: bootstrap persisted config from descriptor defaults * admin/plugin: dedupe scheduled proposals by dedupe key * admin/ui: add job type and state filters for plugin monitoring * admin/ui: add per-job-type plugin activity summary * admin/plugin: split descriptor read API from schema refresh * admin/ui: keep plugin summary metrics global while tables are filtered * admin/plugin: retry executor reservation before timing out * admin/plugin: expose scheduler states for monitoring * admin/ui: show per-job-type scheduler states in plugin monitor * pb/plugin: rename protobuf package to plugin * admin/plugin: rename pluginRuntime wiring to plugin * admin/plugin: remove runtime naming from plugin APIs and UI * admin/plugin: rename runtime files to plugin naming * admin/plugin: persist jobs and activities for monitor recovery * admin/plugin: lease one detector worker per job type * admin/ui: show worker load from plugin heartbeats * admin/plugin: skip stale workers for detector and executor picks * plugin/worker: add plugin worker command and stream runtime scaffold * plugin/worker: implement vacuum detect and execute handlers * admin/plugin: document external vacuum plugin worker starter * command: update plugin.worker help to reflect implemented flow * command/admin: drop legacy Plugin V2 label * plugin/worker: validate vacuum job type and respect min interval * plugin/worker: test no-op detect when min interval not elapsed * command/admin: document plugin.worker external process * plugin/worker: advertise configured concurrency in hello * command/plugin.worker: add jobType handler selection * command/plugin.worker: test handler selection by job type * command/plugin.worker: persist worker id in workingDir * admin/plugin: document plugin.worker jobType and workingDir flags * plugin/worker: support cancel request for in-flight work * plugin/worker: test cancel request acknowledgements * command/plugin.worker: document workingDir and jobType behavior * plugin/worker: emit executor activity events for monitor * plugin/worker: test executor activity builder * admin/plugin: send last successful run in detection request * admin/plugin: send cancel request when detect or execute context ends * admin/plugin: document worker cancel request responsibility * admin/handlers: expose plugin scheduler states API in no-auth mode * admin/handlers: test plugin scheduler states route registration * admin/plugin: keep worker id on worker-generated activity records * admin/plugin: test worker id propagation in monitor activities * admin/dash: always initialize plugin service * command/admin: remove plugin enable flags and default to enabled * admin/dash: drop pluginEnabled constructor parameter * admin/plugin UI: stop checking plugin enabled state * admin/plugin: remove docs for plugin enable flags * admin/dash: remove unused plugin enabled check method * admin/dash: fallback to in-memory plugin init when dataDir fails * admin/plugin API: expose worker gRPC port in status * command/plugin.worker: resolve admin gRPC port via plugin status * split plugin UI into overview/configuration/monitoring pages * Update layout_templ.go * add volume_balance plugin worker handler * wire plugin.worker CLI for volume_balance job type * add erasure_coding plugin worker handler * wire plugin.worker CLI for erasure_coding job type * support multi-job handlers in plugin worker runtime * allow plugin.worker jobType as comma-separated list * admin/plugin UI: rename to Workers and simplify config view * plugin worker: queue detection requests instead of capacity reject * Update plugin_worker.go * plugin volume_balance: remove force_move/timeout from worker config UI * plugin erasure_coding: enforce local working dir and cleanup * admin/plugin UI: rename admin settings to job scheduling * admin/plugin UI: persist and robustly render detection results * admin/plugin: record and return detection trace metadata * admin/plugin UI: show detection process and decision trace * plugin: surface detector decision trace as activities * mini: start a plugin worker by default * admin/plugin UI: split monitoring into detection and execution tabs * plugin worker: emit detection decision trace for EC and balance * admin workers UI: split monitoring into detection and execution pages * plugin scheduler: skip proposals for active assigned/running jobs * admin workers UI: add job queue tab * plugin worker: add dummy stress detector and executor job type * admin workers UI: reorder tabs to detection queue execution * admin workers UI: regenerate plugin template * plugin defaults: include dummy stress and add stress tests * plugin dummy stress: rotate detection selections across runs * plugin scheduler: remove cross-run proposal dedupe * plugin queue: track pending scheduled jobs * plugin scheduler: wait for executor capacity before dispatch * plugin scheduler: skip detection when waiting backlog is high * plugin: add disk-backed job detail API and persistence * admin ui: show plugin job detail modal from job id links * plugin: generate unique job ids instead of reusing proposal ids * plugin worker: emit heartbeats on work state changes * plugin registry: round-robin tied executor and detector picks * add temporary EC overnight stress runner * plugin job details: persist and render EC execution plans * ec volume details: color data and parity shard badges * shard labels: keep parity ids numeric and color-only distinction * admin: remove legacy maintenance UI routes and templates * admin: remove dead maintenance endpoint helpers * Update layout_templ.go * remove dummy_stress worker and command support * refactor plugin UI to job-type top tabs and sub-tabs * migrate weed worker command to plugin runtime * remove plugin.worker command and keep worker runtime with metrics * update helm worker args for jobType and execution flags * set plugin scheduling defaults to global 16 and per-worker 4 * stress: fix RPC context reuse and remove redundant variables in ec_stress_runner * admin/plugin: fix lifecycle races, safe channel operations, and terminal state constants * admin/dash: randomize job IDs and fix priority zero-value overwrite in plugin API * admin/handlers: implement buffered rendering to prevent response corruption * admin/plugin: implement debounced persistence flusher and optimize BuildJobDetail memory lookups * admin/plugin: fix priority overwrite and implement bounded wait in scheduler reserve * admin/plugin: implement atomic file writes and fix run record side effects * admin/plugin: use P prefix for parity shard labels in execution plans * admin/plugin: enable parallel execution for cancellation tests * admin: refactor time.Time fields to pointers for better JSON omitempty support * admin/plugin: implement pointer-safe time assignments and comparisons in plugin core * admin/plugin: fix time assignment and sorting logic in plugin monitor after pointer refactor * admin/plugin: update scheduler activity tracking to use time pointers * admin/plugin: fix time-based run history trimming after pointer refactor * admin/dash: fix JobSpec struct literal in plugin API after pointer refactor * admin/view: add D/P prefixes to EC shard badges for UI consistency * admin/plugin: use lifecycle-aware context for schema prefetching * Update ec_volume_details_templ.go * admin/stress: fix proposal sorting and log volume cleanup errors * stress: refine ec stress runner with math/rand and collection name - Added Collection field to VolumeEcShardsDeleteRequest for correct filename construction. - Replaced crypto/rand with seeded math/rand PRNG for bulk payloads. - Added documentation for EcMinAge zero-value behavior. - Added logging for ignored errors in volume/shard deletion. * admin: return internal server error for plugin store failures Changed error status code from 400 Bad Request to 500 Internal Server Error for failures in GetPluginJobDetail to correctly reflect server-side errors. * admin: implement safe channel sends and graceful shutdown sync - Added sync.WaitGroup to Plugin struct to manage background goroutines. - Implemented safeSendCh helper using recover() to prevent panics on closed channels. - Ensured Shutdown() waits for all background operations to complete. * admin: robustify plugin monitor with nil-safe time and record init - Standardized nil-safe assignment for *time.Time pointers (CreatedAt, UpdatedAt, CompletedAt). - Ensured persistJobDetailSnapshot initializes new records correctly if they don't exist on disk. - Fixed debounced persistence to trigger immediate write on job completion. * admin: improve scheduler shutdown behavior and logic guards - Replaced brittle error string matching with explicit r.shutdownCh selection for shutdown detection. - Removed redundant nil guard in buildScheduledJobSpec. - Standardized WaitGroup usage for schedulerLoop. * admin: implement deep copy for job parameters and atomic write fixes - Implemented deepCopyGenericValue and used it in cloneTrackedJob to prevent shared state. - Ensured atomicWriteFile creates parent directories before writing. * admin: remove unreachable branch in shard classification Removed an unreachable 'totalShards <= 0' check in classifyShardID as dataShards and parityShards are already guarded. * admin: secure UI links and use canonical shard constants - Added rel="noopener noreferrer" to external links for security. - Replaced magic number 14 with erasure_coding.TotalShardsCount. - Used renderEcShardBadge for missing shard list consistency. * admin: stabilize plugin tests and fix regressions - Composed a robust plugin_monitor_test.go to handle asynchronous persistence. - Updated all time.Time literals to use timeToPtr helper. - Added explicit Shutdown() calls in tests to synchronize with debounced writes. - Fixed syntax errors and orphaned struct literals in tests. * Potential fix for code scanning alert no. 278: Slice memory allocation with excessive size value Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com> * Potential fix for code scanning alert no. 283: Uncontrolled data used in path expression Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com> * admin: finalize refinements for error handling, scheduler, and race fixes - Standardized HTTP 500 status codes for store failures in plugin_api.go. - Tracked scheduled detection goroutines with sync.WaitGroup for safe shutdown. - Fixed race condition in safeSendDetectionComplete by extracting channel under lock. - Implemented deep copy for JobActivity details. - Used defaultDirPerm constant in atomicWriteFile. * test(ec): migrate admin dockertest to plugin APIs * admin/plugin_api: fix RunPluginJobTypeAPI to return 500 for server-side detection/filter errors * admin/plugin_api: fix ExecutePluginJobAPI to return 500 for job execution failures * admin/plugin_api: limit parseProtoJSONBody request body to 1MB to prevent unbounded memory usage * admin/plugin: consolidate regex to package-level validJobTypePattern; add char validation to sanitizeJobID * admin/plugin: fix racy Shutdown channel close with sync.Once * admin/plugin: track sendLoop and recv goroutines in WorkerStream with r.wg * admin/plugin: document writeProtoFiles atomicity — .pb is source of truth, .json is human-readable only * admin/plugin: extract activityLess helper to deduplicate nil-safe OccurredAt sort comparators * test/ec: check http.NewRequest errors to prevent nil req panics * test/ec: replace deprecated ioutil/math/rand, fix stale step comment 5.1→3.1 * plugin(ec): raise default detection and scheduling throughput limits * topology: include empty disks in volume list and EC capacity fallback * topology: remove hard 10-task cap for detection planning * Update ec_volume_details_templ.go * adjust default * fix tests --------- Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
This commit is contained in:
Chris Lu
@@ -14,6 +14,8 @@ gen:
|
||||
protoc mq_schema.proto --go_out=./schema_pb --go-grpc_out=./schema_pb --go_opt=paths=source_relative --go-grpc_opt=paths=source_relative
|
||||
protoc mq_agent.proto --go_out=./mq_agent_pb --go-grpc_out=./mq_agent_pb --go_opt=paths=source_relative --go-grpc_opt=paths=source_relative
|
||||
protoc worker.proto --go_out=./worker_pb --go-grpc_out=./worker_pb --go_opt=paths=source_relative --go-grpc_opt=paths=source_relative
|
||||
mkdir -p ./plugin_pb
|
||||
protoc plugin.proto --go_out=./plugin_pb --go-grpc_out=./plugin_pb --go_opt=paths=source_relative --go-grpc_opt=paths=source_relative
|
||||
# protoc filer.proto --java_out=../../other/java/client/src/main/java
|
||||
cp filer.proto ../../other/java/client/src/main/proto
|
||||
|
||||
|
||||
443
weed/pb/plugin.proto
Normal file
443
weed/pb/plugin.proto
Normal file
@@ -0,0 +1,443 @@
|
||||
syntax = "proto3";
|
||||
|
||||
package plugin;
|
||||
|
||||
option go_package = "github.com/seaweedfs/seaweedfs/weed/pb/plugin_pb";
|
||||
|
||||
import "google/protobuf/duration.proto";
|
||||
import "google/protobuf/timestamp.proto";
|
||||
|
||||
// PluginControlService is the admin-facing stream API for external workers.
|
||||
// Workers initiate and keep this stream alive; all control plane traffic flows through it.
|
||||
service PluginControlService {
|
||||
rpc WorkerStream(stream WorkerToAdminMessage) returns (stream AdminToWorkerMessage);
|
||||
}
|
||||
|
||||
// WorkerToAdminMessage carries worker-originated events and responses.
|
||||
message WorkerToAdminMessage {
|
||||
string worker_id = 1;
|
||||
google.protobuf.Timestamp sent_at = 2;
|
||||
|
||||
oneof body {
|
||||
WorkerHello hello = 10;
|
||||
WorkerHeartbeat heartbeat = 11;
|
||||
WorkerAcknowledge acknowledge = 12;
|
||||
ConfigSchemaResponse config_schema_response = 13;
|
||||
DetectionProposals detection_proposals = 14;
|
||||
DetectionComplete detection_complete = 15;
|
||||
JobProgressUpdate job_progress_update = 16;
|
||||
JobCompleted job_completed = 17;
|
||||
}
|
||||
}
|
||||
|
||||
// AdminToWorkerMessage carries commands and lifecycle notifications from admin.
|
||||
message AdminToWorkerMessage {
|
||||
string request_id = 1;
|
||||
google.protobuf.Timestamp sent_at = 2;
|
||||
|
||||
oneof body {
|
||||
AdminHello hello = 10;
|
||||
RequestConfigSchema request_config_schema = 11;
|
||||
RunDetectionRequest run_detection_request = 12;
|
||||
ExecuteJobRequest execute_job_request = 13;
|
||||
CancelRequest cancel_request = 14;
|
||||
AdminShutdown shutdown = 15;
|
||||
}
|
||||
}
|
||||
|
||||
message WorkerHello {
|
||||
string worker_id = 1;
|
||||
string worker_instance_id = 2;
|
||||
string address = 3;
|
||||
string worker_version = 4;
|
||||
string protocol_version = 5;
|
||||
repeated JobTypeCapability capabilities = 6;
|
||||
map<string, string> metadata = 7;
|
||||
}
|
||||
|
||||
message AdminHello {
|
||||
bool accepted = 1;
|
||||
string message = 2;
|
||||
int32 heartbeat_interval_seconds = 3;
|
||||
int32 reconnect_delay_seconds = 4;
|
||||
}
|
||||
|
||||
message WorkerHeartbeat {
|
||||
string worker_id = 1;
|
||||
repeated RunningWork running_work = 2;
|
||||
int32 detection_slots_used = 3;
|
||||
int32 detection_slots_total = 4;
|
||||
int32 execution_slots_used = 5;
|
||||
int32 execution_slots_total = 6;
|
||||
map<string, int32> queued_jobs_by_type = 7;
|
||||
map<string, string> metadata = 8;
|
||||
}
|
||||
|
||||
message WorkerAcknowledge {
|
||||
string request_id = 1;
|
||||
bool accepted = 2;
|
||||
string message = 3;
|
||||
}
|
||||
|
||||
message RunningWork {
|
||||
string work_id = 1;
|
||||
WorkKind kind = 2;
|
||||
string job_type = 3;
|
||||
JobState state = 4;
|
||||
double progress_percent = 5;
|
||||
string stage = 6;
|
||||
}
|
||||
|
||||
message JobTypeCapability {
|
||||
string job_type = 1;
|
||||
bool can_detect = 2;
|
||||
bool can_execute = 3;
|
||||
int32 max_detection_concurrency = 4;
|
||||
int32 max_execution_concurrency = 5;
|
||||
string display_name = 6;
|
||||
string description = 7;
|
||||
}
|
||||
|
||||
message RequestConfigSchema {
|
||||
string job_type = 1;
|
||||
bool force_refresh = 2;
|
||||
}
|
||||
|
||||
message ConfigSchemaResponse {
|
||||
string request_id = 1;
|
||||
string job_type = 2;
|
||||
bool success = 3;
|
||||
string error_message = 4;
|
||||
JobTypeDescriptor job_type_descriptor = 5;
|
||||
}
|
||||
|
||||
// JobTypeDescriptor defines one job type contract, including UI schema and defaults.
|
||||
message JobTypeDescriptor {
|
||||
string job_type = 1;
|
||||
string display_name = 2;
|
||||
string description = 3;
|
||||
string icon = 4;
|
||||
uint32 descriptor_version = 5;
|
||||
|
||||
// Admin-owned options such as detection frequency and dispatch concurrency.
|
||||
ConfigForm admin_config_form = 6;
|
||||
// Worker-owned options used during detection and execution.
|
||||
ConfigForm worker_config_form = 7;
|
||||
|
||||
AdminRuntimeDefaults admin_runtime_defaults = 8;
|
||||
map<string, ConfigValue> worker_default_values = 9;
|
||||
}
|
||||
|
||||
message ConfigForm {
|
||||
string form_id = 1;
|
||||
string title = 2;
|
||||
string description = 3;
|
||||
repeated ConfigSection sections = 4;
|
||||
map<string, ConfigValue> default_values = 5;
|
||||
}
|
||||
|
||||
message ConfigSection {
|
||||
string section_id = 1;
|
||||
string title = 2;
|
||||
string description = 3;
|
||||
repeated ConfigField fields = 4;
|
||||
}
|
||||
|
||||
message ConfigField {
|
||||
string name = 1;
|
||||
string label = 2;
|
||||
string description = 3;
|
||||
string help_text = 4;
|
||||
string placeholder = 5;
|
||||
|
||||
ConfigFieldType field_type = 6;
|
||||
ConfigWidget widget = 7;
|
||||
|
||||
bool required = 8;
|
||||
bool read_only = 9;
|
||||
bool sensitive = 10;
|
||||
|
||||
ConfigValue min_value = 11;
|
||||
ConfigValue max_value = 12;
|
||||
|
||||
repeated ConfigOption options = 13;
|
||||
repeated ValidationRule validation_rules = 14;
|
||||
|
||||
// Simple visibility dependency: show this field when the referenced field equals value.
|
||||
string visible_when_field = 15;
|
||||
ConfigValue visible_when_equals = 16;
|
||||
}
|
||||
|
||||
message ConfigOption {
|
||||
string value = 1;
|
||||
string label = 2;
|
||||
string description = 3;
|
||||
bool disabled = 4;
|
||||
}
|
||||
|
||||
message ValidationRule {
|
||||
ValidationRuleType type = 1;
|
||||
string expression = 2;
|
||||
string error_message = 3;
|
||||
}
|
||||
|
||||
message ConfigValue {
|
||||
oneof kind {
|
||||
bool bool_value = 1;
|
||||
int64 int64_value = 2;
|
||||
double double_value = 3;
|
||||
string string_value = 4;
|
||||
bytes bytes_value = 5;
|
||||
google.protobuf.Duration duration_value = 6;
|
||||
StringList string_list = 7;
|
||||
Int64List int64_list = 8;
|
||||
DoubleList double_list = 9;
|
||||
BoolList bool_list = 10;
|
||||
ValueList list_value = 11;
|
||||
ValueMap map_value = 12;
|
||||
}
|
||||
}
|
||||
|
||||
message StringList {
|
||||
repeated string values = 1;
|
||||
}
|
||||
|
||||
message Int64List {
|
||||
repeated int64 values = 1;
|
||||
}
|
||||
|
||||
message DoubleList {
|
||||
repeated double values = 1;
|
||||
}
|
||||
|
||||
message BoolList {
|
||||
repeated bool values = 1;
|
||||
}
|
||||
|
||||
message ValueList {
|
||||
repeated ConfigValue values = 1;
|
||||
}
|
||||
|
||||
message ValueMap {
|
||||
map<string, ConfigValue> fields = 1;
|
||||
}
|
||||
|
||||
message AdminRuntimeDefaults {
|
||||
bool enabled = 1;
|
||||
int32 detection_interval_seconds = 2;
|
||||
int32 detection_timeout_seconds = 3;
|
||||
int32 max_jobs_per_detection = 4;
|
||||
int32 global_execution_concurrency = 5;
|
||||
int32 per_worker_execution_concurrency = 6;
|
||||
int32 retry_limit = 7;
|
||||
int32 retry_backoff_seconds = 8;
|
||||
}
|
||||
|
||||
message AdminRuntimeConfig {
|
||||
bool enabled = 1;
|
||||
int32 detection_interval_seconds = 2;
|
||||
int32 detection_timeout_seconds = 3;
|
||||
int32 max_jobs_per_detection = 4;
|
||||
int32 global_execution_concurrency = 5;
|
||||
int32 per_worker_execution_concurrency = 6;
|
||||
int32 retry_limit = 7;
|
||||
int32 retry_backoff_seconds = 8;
|
||||
}
|
||||
|
||||
message RunDetectionRequest {
|
||||
string request_id = 1;
|
||||
string job_type = 2;
|
||||
int64 detection_sequence = 3;
|
||||
|
||||
AdminRuntimeConfig admin_runtime = 4;
|
||||
map<string, ConfigValue> admin_config_values = 5;
|
||||
map<string, ConfigValue> worker_config_values = 6;
|
||||
|
||||
ClusterContext cluster_context = 7;
|
||||
google.protobuf.Timestamp last_successful_run = 8;
|
||||
int32 max_results = 9;
|
||||
}
|
||||
|
||||
message DetectionProposals {
|
||||
string request_id = 1;
|
||||
string job_type = 2;
|
||||
repeated JobProposal proposals = 3;
|
||||
bool has_more = 4;
|
||||
}
|
||||
|
||||
message DetectionComplete {
|
||||
string request_id = 1;
|
||||
string job_type = 2;
|
||||
bool success = 3;
|
||||
string error_message = 4;
|
||||
int32 total_proposals = 5;
|
||||
}
|
||||
|
||||
message JobProposal {
|
||||
string proposal_id = 1;
|
||||
string dedupe_key = 2;
|
||||
string job_type = 3;
|
||||
JobPriority priority = 4;
|
||||
string summary = 5;
|
||||
string detail = 6;
|
||||
map<string, ConfigValue> parameters = 7;
|
||||
map<string, string> labels = 8;
|
||||
google.protobuf.Timestamp not_before = 9;
|
||||
google.protobuf.Timestamp expires_at = 10;
|
||||
}
|
||||
|
||||
message ExecuteJobRequest {
|
||||
string request_id = 1;
|
||||
JobSpec job = 2;
|
||||
AdminRuntimeConfig admin_runtime = 3;
|
||||
map<string, ConfigValue> admin_config_values = 4;
|
||||
map<string, ConfigValue> worker_config_values = 5;
|
||||
ClusterContext cluster_context = 6;
|
||||
int32 attempt = 7;
|
||||
}
|
||||
|
||||
message JobSpec {
|
||||
string job_id = 1;
|
||||
string job_type = 2;
|
||||
string dedupe_key = 3;
|
||||
JobPriority priority = 4;
|
||||
string summary = 5;
|
||||
string detail = 6;
|
||||
map<string, ConfigValue> parameters = 7;
|
||||
map<string, string> labels = 8;
|
||||
google.protobuf.Timestamp created_at = 9;
|
||||
google.protobuf.Timestamp scheduled_at = 10;
|
||||
}
|
||||
|
||||
message JobProgressUpdate {
|
||||
string request_id = 1;
|
||||
string job_id = 2;
|
||||
string job_type = 3;
|
||||
JobState state = 4;
|
||||
double progress_percent = 5;
|
||||
string stage = 6;
|
||||
string message = 7;
|
||||
map<string, ConfigValue> metrics = 8;
|
||||
repeated ActivityEvent activities = 9;
|
||||
google.protobuf.Timestamp updated_at = 10;
|
||||
}
|
||||
|
||||
message JobCompleted {
|
||||
string request_id = 1;
|
||||
string job_id = 2;
|
||||
string job_type = 3;
|
||||
bool success = 4;
|
||||
string error_message = 5;
|
||||
JobResult result = 6;
|
||||
repeated ActivityEvent activities = 7;
|
||||
google.protobuf.Timestamp completed_at = 8;
|
||||
}
|
||||
|
||||
message JobResult {
|
||||
map<string, ConfigValue> output_values = 1;
|
||||
string summary = 2;
|
||||
}
|
||||
|
||||
message ClusterContext {
|
||||
repeated string master_grpc_addresses = 1;
|
||||
repeated string filer_grpc_addresses = 2;
|
||||
repeated string volume_grpc_addresses = 3;
|
||||
map<string, string> metadata = 4;
|
||||
}
|
||||
|
||||
message ActivityEvent {
|
||||
ActivitySource source = 1;
|
||||
string message = 2;
|
||||
string stage = 3;
|
||||
map<string, ConfigValue> details = 4;
|
||||
google.protobuf.Timestamp created_at = 5;
|
||||
}
|
||||
|
||||
message CancelRequest {
|
||||
string target_id = 1;
|
||||
WorkKind target_kind = 2;
|
||||
string reason = 3;
|
||||
bool force = 4;
|
||||
}
|
||||
|
||||
message AdminShutdown {
|
||||
string reason = 1;
|
||||
int32 grace_period_seconds = 2;
|
||||
}
|
||||
|
||||
// PersistedJobTypeConfig is the admin-side on-disk model per job type.
|
||||
message PersistedJobTypeConfig {
|
||||
string job_type = 1;
|
||||
uint32 descriptor_version = 2;
|
||||
map<string, ConfigValue> admin_config_values = 3;
|
||||
map<string, ConfigValue> worker_config_values = 4;
|
||||
AdminRuntimeConfig admin_runtime = 5;
|
||||
google.protobuf.Timestamp updated_at = 6;
|
||||
string updated_by = 7;
|
||||
}
|
||||
|
||||
enum WorkKind {
|
||||
WORK_KIND_UNSPECIFIED = 0;
|
||||
WORK_KIND_DETECTION = 1;
|
||||
WORK_KIND_EXECUTION = 2;
|
||||
}
|
||||
|
||||
enum JobPriority {
|
||||
JOB_PRIORITY_UNSPECIFIED = 0;
|
||||
JOB_PRIORITY_LOW = 1;
|
||||
JOB_PRIORITY_NORMAL = 2;
|
||||
JOB_PRIORITY_HIGH = 3;
|
||||
JOB_PRIORITY_CRITICAL = 4;
|
||||
}
|
||||
|
||||
enum JobState {
|
||||
JOB_STATE_UNSPECIFIED = 0;
|
||||
JOB_STATE_PENDING = 1;
|
||||
JOB_STATE_ASSIGNED = 2;
|
||||
JOB_STATE_RUNNING = 3;
|
||||
JOB_STATE_SUCCEEDED = 4;
|
||||
JOB_STATE_FAILED = 5;
|
||||
JOB_STATE_CANCELED = 6;
|
||||
}
|
||||
|
||||
enum ConfigFieldType {
|
||||
CONFIG_FIELD_TYPE_UNSPECIFIED = 0;
|
||||
CONFIG_FIELD_TYPE_BOOL = 1;
|
||||
CONFIG_FIELD_TYPE_INT64 = 2;
|
||||
CONFIG_FIELD_TYPE_DOUBLE = 3;
|
||||
CONFIG_FIELD_TYPE_STRING = 4;
|
||||
CONFIG_FIELD_TYPE_BYTES = 5;
|
||||
CONFIG_FIELD_TYPE_DURATION = 6;
|
||||
CONFIG_FIELD_TYPE_ENUM = 7;
|
||||
CONFIG_FIELD_TYPE_LIST = 8;
|
||||
CONFIG_FIELD_TYPE_OBJECT = 9;
|
||||
}
|
||||
|
||||
enum ConfigWidget {
|
||||
CONFIG_WIDGET_UNSPECIFIED = 0;
|
||||
CONFIG_WIDGET_TOGGLE = 1;
|
||||
CONFIG_WIDGET_TEXT = 2;
|
||||
CONFIG_WIDGET_TEXTAREA = 3;
|
||||
CONFIG_WIDGET_NUMBER = 4;
|
||||
CONFIG_WIDGET_SELECT = 5;
|
||||
CONFIG_WIDGET_MULTI_SELECT = 6;
|
||||
CONFIG_WIDGET_DURATION = 7;
|
||||
CONFIG_WIDGET_PASSWORD = 8;
|
||||
}
|
||||
|
||||
enum ValidationRuleType {
|
||||
VALIDATION_RULE_TYPE_UNSPECIFIED = 0;
|
||||
VALIDATION_RULE_TYPE_REGEX = 1;
|
||||
VALIDATION_RULE_TYPE_MIN_LENGTH = 2;
|
||||
VALIDATION_RULE_TYPE_MAX_LENGTH = 3;
|
||||
VALIDATION_RULE_TYPE_MIN_ITEMS = 4;
|
||||
VALIDATION_RULE_TYPE_MAX_ITEMS = 5;
|
||||
VALIDATION_RULE_TYPE_CUSTOM = 6;
|
||||
}
|
||||
|
||||
enum ActivitySource {
|
||||
ACTIVITY_SOURCE_UNSPECIFIED = 0;
|
||||
ACTIVITY_SOURCE_ADMIN = 1;
|
||||
ACTIVITY_SOURCE_DETECTOR = 2;
|
||||
ACTIVITY_SOURCE_EXECUTOR = 3;
|
||||
}
|
||||
4566
weed/pb/plugin_pb/plugin.pb.go
Normal file
4566
weed/pb/plugin_pb/plugin.pb.go
Normal file
File diff suppressed because it is too large
Load Diff
121
weed/pb/plugin_pb/plugin_grpc.pb.go
Normal file
121
weed/pb/plugin_pb/plugin_grpc.pb.go
Normal file
@@ -0,0 +1,121 @@
|
||||
// Code generated by protoc-gen-go-grpc. DO NOT EDIT.
|
||||
// versions:
|
||||
// - protoc-gen-go-grpc v1.5.1
|
||||
// - protoc v6.33.4
|
||||
// source: plugin.proto
|
||||
|
||||
package plugin_pb
|
||||
|
||||
import (
|
||||
context "context"
|
||||
grpc "google.golang.org/grpc"
|
||||
codes "google.golang.org/grpc/codes"
|
||||
status "google.golang.org/grpc/status"
|
||||
)
|
||||
|
||||
// This is a compile-time assertion to ensure that this generated file
|
||||
// is compatible with the grpc package it is being compiled against.
|
||||
// Requires gRPC-Go v1.64.0 or later.
|
||||
const _ = grpc.SupportPackageIsVersion9
|
||||
|
||||
const (
|
||||
PluginControlService_WorkerStream_FullMethodName = "/plugin.PluginControlService/WorkerStream"
|
||||
)
|
||||
|
||||
// PluginControlServiceClient is the client API for PluginControlService service.
|
||||
//
|
||||
// For semantics around ctx use and closing/ending streaming RPCs, please refer to https://pkg.go.dev/google.golang.org/grpc/?tab=doc#ClientConn.NewStream.
|
||||
//
|
||||
// PluginControlService is the admin-facing stream API for external workers.
|
||||
// Workers initiate and keep this stream alive; all control plane traffic flows through it.
|
||||
type PluginControlServiceClient interface {
|
||||
WorkerStream(ctx context.Context, opts ...grpc.CallOption) (grpc.BidiStreamingClient[WorkerToAdminMessage, AdminToWorkerMessage], error)
|
||||
}
|
||||
|
||||
type pluginControlServiceClient struct {
|
||||
cc grpc.ClientConnInterface
|
||||
}
|
||||
|
||||
func NewPluginControlServiceClient(cc grpc.ClientConnInterface) PluginControlServiceClient {
|
||||
return &pluginControlServiceClient{cc}
|
||||
}
|
||||
|
||||
func (c *pluginControlServiceClient) WorkerStream(ctx context.Context, opts ...grpc.CallOption) (grpc.BidiStreamingClient[WorkerToAdminMessage, AdminToWorkerMessage], error) {
|
||||
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
|
||||
stream, err := c.cc.NewStream(ctx, &PluginControlService_ServiceDesc.Streams[0], PluginControlService_WorkerStream_FullMethodName, cOpts...)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
x := &grpc.GenericClientStream[WorkerToAdminMessage, AdminToWorkerMessage]{ClientStream: stream}
|
||||
return x, nil
|
||||
}
|
||||
|
||||
// This type alias is provided for backwards compatibility with existing code that references the prior non-generic stream type by name.
|
||||
type PluginControlService_WorkerStreamClient = grpc.BidiStreamingClient[WorkerToAdminMessage, AdminToWorkerMessage]
|
||||
|
||||
// PluginControlServiceServer is the server API for PluginControlService service.
|
||||
// All implementations must embed UnimplementedPluginControlServiceServer
|
||||
// for forward compatibility.
|
||||
//
|
||||
// PluginControlService is the admin-facing stream API for external workers.
|
||||
// Workers initiate and keep this stream alive; all control plane traffic flows through it.
|
||||
type PluginControlServiceServer interface {
|
||||
WorkerStream(grpc.BidiStreamingServer[WorkerToAdminMessage, AdminToWorkerMessage]) error
|
||||
mustEmbedUnimplementedPluginControlServiceServer()
|
||||
}
|
||||
|
||||
// UnimplementedPluginControlServiceServer must be embedded to have
|
||||
// forward compatible implementations.
|
||||
//
|
||||
// NOTE: this should be embedded by value instead of pointer to avoid a nil
|
||||
// pointer dereference when methods are called.
|
||||
type UnimplementedPluginControlServiceServer struct{}
|
||||
|
||||
func (UnimplementedPluginControlServiceServer) WorkerStream(grpc.BidiStreamingServer[WorkerToAdminMessage, AdminToWorkerMessage]) error {
|
||||
return status.Errorf(codes.Unimplemented, "method WorkerStream not implemented")
|
||||
}
|
||||
func (UnimplementedPluginControlServiceServer) mustEmbedUnimplementedPluginControlServiceServer() {}
|
||||
func (UnimplementedPluginControlServiceServer) testEmbeddedByValue() {}
|
||||
|
||||
// UnsafePluginControlServiceServer may be embedded to opt out of forward compatibility for this service.
|
||||
// Use of this interface is not recommended, as added methods to PluginControlServiceServer will
|
||||
// result in compilation errors.
|
||||
type UnsafePluginControlServiceServer interface {
|
||||
mustEmbedUnimplementedPluginControlServiceServer()
|
||||
}
|
||||
|
||||
func RegisterPluginControlServiceServer(s grpc.ServiceRegistrar, srv PluginControlServiceServer) {
|
||||
// If the following call pancis, it indicates UnimplementedPluginControlServiceServer was
|
||||
// embedded by pointer and is nil. This will cause panics if an
|
||||
// unimplemented method is ever invoked, so we test this at initialization
|
||||
// time to prevent it from happening at runtime later due to I/O.
|
||||
if t, ok := srv.(interface{ testEmbeddedByValue() }); ok {
|
||||
t.testEmbeddedByValue()
|
||||
}
|
||||
s.RegisterService(&PluginControlService_ServiceDesc, srv)
|
||||
}
|
||||
|
||||
func _PluginControlService_WorkerStream_Handler(srv interface{}, stream grpc.ServerStream) error {
|
||||
return srv.(PluginControlServiceServer).WorkerStream(&grpc.GenericServerStream[WorkerToAdminMessage, AdminToWorkerMessage]{ServerStream: stream})
|
||||
}
|
||||
|
||||
// This type alias is provided for backwards compatibility with existing code that references the prior non-generic stream type by name.
|
||||
type PluginControlService_WorkerStreamServer = grpc.BidiStreamingServer[WorkerToAdminMessage, AdminToWorkerMessage]
|
||||
|
||||
// PluginControlService_ServiceDesc is the grpc.ServiceDesc for PluginControlService service.
|
||||
// It's only intended for direct use with grpc.RegisterService,
|
||||
// and not to be introspected or modified (even as a copy)
|
||||
var PluginControlService_ServiceDesc = grpc.ServiceDesc{
|
||||
ServiceName: "plugin.PluginControlService",
|
||||
HandlerType: (*PluginControlServiceServer)(nil),
|
||||
Methods: []grpc.MethodDesc{},
|
||||
Streams: []grpc.StreamDesc{
|
||||
{
|
||||
StreamName: "WorkerStream",
|
||||
Handler: _PluginControlService_WorkerStream_Handler,
|
||||
ServerStreams: true,
|
||||
ClientStreams: true,
|
||||
},
|
||||
},
|
||||
Metadata: "plugin.proto",
|
||||
}
|
||||
Reference in New Issue
Block a user