Chris Lu 7eb90fdfd7 Enhance EC balancing to separate parity and data shards (#8038) ...

* Enhance EC balancing to separate parity and data shards across racks

* Rename avoidRacks to antiAffinityRacks for clarity

* Implement server-level EC separation for parity/data shards

* Optimize EC balancing: consolidate helpers and extract two-pass selection logic

* Add comprehensive edge case tests for EC balancing logic

* Apply code review feedback: rename select_(), add divide-by-zero guard, fix comment

* Remove unused parameters from doBalanceEcShardsWithinOneRack and add explicit anti-affinity check

* Add disk-level anti-affinity for data/parity shard separation

- Modified pickBestDiskOnNode to accept shardId and dataShardCount
- Implemented explicit anti-affinity: 1000-point penalty for placing data shards on disks with parity (and vice versa)
- Updated all call sites including balancing and evacuation
- For evacuation, disabled anti-affinity by passing dataShardCount=0

2026-01-15 12:43:44 -08:00

..

command_cluster_check.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

command_cluster_ps.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

command_cluster_raft_add.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

command_cluster_raft_leader_transfer_test.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

command_cluster_raft_leader_transfer.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

command_cluster_raft_ps.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

command_cluster_raft_remove.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

command_cluster_status_test.go

Give cluster.status detailed file metrics for regular volumes (#7791)

2025-12-17 16:40:27 -08:00

command_cluster_status.go

opt: reduce ShardsInfo memory usage with bitmap and sorted slice (#7974)

2026-01-06 00:09:52 -08:00

command_collection_delete.go

Unify the parameter to disable dry-run on weed shell commands to -apply (instead of -force). (#7450)

2025-11-09 19:58:38 -08:00

command_collection_list.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

command_ec_balance.go

ec: add -diskType flag to EC commands for SSD support (#7607)

2025-12-10 22:42:52 -08:00

command_ec_common_avoid_test.go

Enhance EC balancing to separate parity and data shards (#8038)

2026-01-15 12:43:44 -08:00

command_ec_common_test.go

fix: EC rebalance fails with replica placement 000 (#7812)

2025-12-19 13:29:12 -08:00

command_ec_common.go

Enhance EC balancing to separate parity and data shards (#8038)

2026-01-15 12:43:44 -08:00

command_ec_decode.go

Fix reporting of EC shard sizes from nodes to masters. (#7835)

2025-12-28 19:30:42 -08:00

command_ec_encode.go

Fix reporting of EC shard sizes from nodes to masters. (#7835)

2025-12-28 19:30:42 -08:00

command_ec_rebuild_test.go

avoid extra missing shard warning

2026-01-04 00:38:53 -08:00

command_ec_rebuild.go

opt: reduce ShardsInfo memory usage with bitmap and sorted slice (#7974)

2026-01-06 00:09:52 -08:00

command_ec_test.go

Fix reporting of EC shard sizes from nodes to masters. (#7835)

2025-12-28 19:30:42 -08:00

command_fs_cat.go

Shell: Added a helper function isHelpRequest() (#7380)

2025-10-24 20:21:35 -07:00

command_fs_cd.go

Shell: Added a helper function isHelpRequest() (#7380)

2025-10-24 20:21:35 -07:00

command_fs_configure.go

Unify the parameter to disable dry-run on weed shell commands to -apply (instead of -force). (#7450)

2025-11-09 19:58:38 -08:00

command_fs_du.go

Shell: Added a helper function isHelpRequest() (#7380)

2025-10-24 20:21:35 -07:00

command_fs_log.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

command_fs_ls.go

Shell: Added a helper function isHelpRequest() (#7380)

2025-10-24 20:21:35 -07:00

command_fs_merge_volumes.go

Fix volume.fsck -forcePurging -reallyDeleteFromVolume to fail fast on filer traversal errors (#8015)

2026-01-14 21:37:50 -08:00

command_fs_meta_cat.go

Shell: Added a helper function isHelpRequest() (#7380)

2025-10-24 20:21:35 -07:00

command_fs_meta_change_volume_id.go

Fix volume.fsck -forcePurging -reallyDeleteFromVolume to fail fast on filer traversal errors (#8015)

2026-01-14 21:37:50 -08:00

command_fs_meta_load.go

de/compress the fs meta file if filename ends with gz/gzip (#7500)

2025-11-18 07:45:12 -08:00

command_fs_meta_notify.go

Fix volume.fsck -forcePurging -reallyDeleteFromVolume to fail fast on filer traversal errors (#8015)

2026-01-14 21:37:50 -08:00

command_fs_meta_save.go

Fix volume.fsck -forcePurging -reallyDeleteFromVolume to fail fast on filer traversal errors (#8015)

2026-01-14 21:37:50 -08:00

command_fs_mkdir.go

Shell: Added a helper function isHelpRequest() (#7380)

2025-10-24 20:21:35 -07:00

command_fs_mv.go

Shell: Added a helper function isHelpRequest() (#7380)

2025-10-24 20:21:35 -07:00

command_fs_pwd.go

Shell: Added a helper function isHelpRequest() (#7380)

2025-10-24 20:21:35 -07:00

command_fs_rm.go

Shell: Added a helper function isHelpRequest() (#7380)

2025-10-24 20:21:35 -07:00

command_fs_tree.go

Shell: Added a helper function isHelpRequest() (#7380)

2025-10-24 20:21:35 -07:00

command_fs_verify.go

Fix volume.fsck -forcePurging -reallyDeleteFromVolume to fail fast on filer traversal errors (#8015)

2026-01-14 21:37:50 -08:00

command_lock_unlock.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

command_mount_configure.go

Clean up logs and deprecated functions (#7339)

2025-10-17 22:11:50 -07:00

command_mq_balance.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

command_mq_topic_compact.go

Add Kafka Gateway (#7231)

2025-10-13 18:05:17 -07:00

command_mq_topic_configure.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

command_mq_topic_desc.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

command_mq_topic_list.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

command_mq_topic_truncate.go

Message Queue: Add sql querying (#7185)

2025-09-09 01:01:03 -07:00

command_remote_cache.go

s3: fix remote object not caching (#7790)

2025-12-16 12:41:04 -08:00

command_remote_configure.go

add one example

2025-12-28 11:39:06 -08:00

command_remote_copy_local.go

Add remote.copy.local command to copy local files to remote storage (#8033)

2026-01-15 00:52:57 -08:00

command_remote_meta_sync.go

Fix remote.meta.sync TTL issue (#8021) (#8030)

2026-01-14 14:45:52 -08:00

command_remote_mount_buckets.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

command_remote_mount.go

Fix remote.meta.sync TTL issue (#8021) (#8030)

2026-01-14 14:45:52 -08:00

command_remote_uncache.go

convert error fromating to %w everywhere (#6995)

2025-07-16 23:39:27 -07:00

command_remote_unmount.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

command_s3_bucket_create.go

shell: add -owner flag to s3.bucket.create command (#7728)

2025-12-12 18:06:13 -08:00

command_s3_bucket_delete.go

fix: admin UI bucket delete now properly deletes collection and checks Object Lock (#7734)

2025-12-13 13:41:25 -08:00

command_s3_bucket_list.go

shell: add -owner flag to s3.bucket.create command (#7728)

2025-12-12 18:06:13 -08:00

command_s3_bucket_owner.go

shell: add -owner flag to s3.bucket.create command (#7728)

2025-12-12 18:06:13 -08:00

command_s3_bucket_quota_check.go

Reduce memory allocations in hot paths (#7725)

2025-12-12 12:51:48 -08:00

command_s3_bucket_quota.go

convert error fromating to %w everywhere (#6995)

2025-07-16 23:39:27 -07:00

command_s3_circuitbreaker_test.go

refactor(shell): readability improvements (#3704)

2022-09-16 02:43:49 -07:00

command_s3_circuitbreaker.go

chore(deps): bump gocloud.dev from 0.40.0 to 0.41.0 (#6679)

2025-03-31 21:42:54 -07:00

command_s3_clean_uploads.go

convert error fromating to %w everywhere (#6995)

2025-07-16 23:39:27 -07:00

command_s3_configure.go

Account Info (#7507)

2025-11-19 20:34:38 -08:00

command_volume_balance_test.go

Shell: support regular expression for collection selection (#7158)

2025-08-23 11:04:24 -07:00

command_volume_balance.go

opt: reduce ShardsInfo memory usage with bitmap and sorted slice (#7974)

2026-01-06 00:09:52 -08:00

command_volume_check_disk_test.go

Mutex command output writes for volume.check.disk. (#7605)

2025-12-02 10:14:24 -08:00

command_volume_check_disk.go

Fix handling of fixed read-only volumes for volume.check.disk. (#7612)

2025-12-03 11:33:35 -08:00

command_volume_configure_replication.go

convert error fromating to %w everywhere (#6995)

2025-07-16 23:39:27 -07:00

command_volume_copy.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

command_volume_delete_empty.go

Unify the parameter to disable dry-run on weed shell commands to -apply (instead of -force). (#7450)

2025-11-09 19:58:38 -08:00

command_volume_delete.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

command_volume_fix_replication_test.go

Fix/copy before delete replication (#6064)

2024-09-26 08:34:13 -07:00

command_volume_fix_replication.go

verbose mode

2025-12-26 12:42:00 -08:00

command_volume_fsck.go

Fix volume.fsck -forcePurging -reallyDeleteFromVolume to fail fast on filer traversal errors (#8015)

2026-01-14 21:37:50 -08:00

command_volume_grow.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

command_volume_list_test.go

opt: reduce ShardsInfo memory usage with bitmap and sorted slice (#7974)

2026-01-06 00:09:52 -08:00

command_volume_list.go

Have volume.list account for EC shards when computing disk usage. (#7909)

2025-12-30 17:27:11 -08:00

command_volume_mark.go

refactor

2024-09-29 10:38:22 -07:00

command_volume_mount.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

command_volume_move.go

Parallelize volume replica operations within ec.encode. (#6374)

2024-12-18 11:59:48 -08:00

command_volume_replica_check_test.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

command_volume_replica_check.go

fix: sync replica entries before ec.encode and volume.tier.move (#7798)

2025-12-16 23:16:07 -08:00

command_volume_server_evacuate_test.go

fix tests

2023-03-13 10:00:27 -07:00

command_volume_server_evacuate.go

Enhance EC balancing to separate parity and data shards (#8038)

2026-01-15 12:43:44 -08:00

command_volume_server_leave.go

Unify the parameter to disable dry-run on weed shell commands to -apply (instead of -force). (#7450)

2025-11-09 19:58:38 -08:00

command_volume_tier_download.go

Shell: support regular expression for collection selection (#7158)

2025-08-23 11:04:24 -07:00

command_volume_tier_move.go

fix: sync replica entries before ec.encode and volume.tier.move (#7798)

2025-12-16 23:16:07 -08:00

command_volume_tier_upload.go

Shell: support regular expression for collection selection (#7158)

2025-08-23 11:04:24 -07:00

command_volume_unmount.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

command_volume_vacuum_disable.go

refactor

2024-09-29 10:38:22 -07:00

command_volume_vacuum_enable.go

refactor

2024-09-29 10:38:22 -07:00

command_volume_vacuum.go

refactor

2024-09-29 10:38:22 -07:00

command.go

refactor

2024-09-29 10:38:22 -07:00

commands.go

Fix masterclient vidmap race condition (#7412)

2025-10-30 23:36:06 -07:00

common_test.go

Humanize output for weed.server by default (#7758)

2025-12-15 11:18:45 -08:00

common.go

Parallelize read-only volume check pass for volume.check.disk. (#7602)

2025-12-02 09:29:27 -08:00

ec_proportional_rebalance_test.go

fix: EC rebalance fails with replica placement 000 (#7812)

2025-12-19 13:29:12 -08:00

ec_proportional_rebalance.go

Fix reporting of EC shard sizes from nodes to masters. (#7835)

2025-12-28 19:30:42 -08:00

ec_rebalance_slots_test.go

opt: reduce ShardsInfo memory usage with bitmap and sorted slice (#7974)

2026-01-06 00:09:52 -08:00

shell_liner.go

Store full shell command in shell history (#7378)

2025-10-24 19:02:44 -07:00

volume.ecshards.txt

Humanize output for weed.server by default (#7758)

2025-12-15 11:18:45 -08:00

volume.list2.txt

Humanize output for weed.server by default (#7758)

2025-12-15 11:18:45 -08:00

volume.list.txt

Humanize output for weed.server by default (#7758)

2025-12-15 11:18:45 -08:00