seaweedFS

Files

Chris Lu 7fbdb9b7b7 feat(shell): add volume.tier.compact command to reclaim cloud storage space (#8715 )

* feat(shell): add volume.tier.compact command to reclaim cloud storage space

Adds a new shell command that automates compaction of cloud tier volumes.
When files are deleted from remote-tiered volumes, space is not reclaimed
on the cloud storage. This command orchestrates: download from remote,
compact locally, and re-upload to reclaim deleted space.

Closes #8563

* fix: log cleanup errors in compactVolumeOnServer instead of discarding them

Helps operators diagnose leftover temp files (.cpd/.cpx) if cleanup
fails after a compaction or commit failure.

* fix: return aggregate error from loop and use regex for collection filter

- Track and return error count when one or more volumes fail to compact,
  so callers see partial failures instead of always getting nil.
- Use compileCollectionPattern for -collection in -volumeId mode too, so
  regex patterns work consistently with the flag description. Empty
  pattern (no -collection given) matches all collections.

2026-03-20 23:52:12 -07:00

command_cluster_check.go

chore: execute goimports to format the code (#7983 )

2026-01-07 13:06:08 -08:00

command_cluster_ps.go

chore: execute goimports to format the code (#7983 )

2026-01-07 13:06:08 -08:00

command_cluster_raft_add.go

chore: execute goimports to format the code (#7983 )

2026-01-07 13:06:08 -08:00

command_cluster_raft_leader_transfer_test.go

chore: execute goimports to format the code (#7983 )

2026-01-07 13:06:08 -08:00

command_cluster_raft_leader_transfer.go

chore: execute goimports to format the code (#7983 )

2026-01-07 13:06:08 -08:00

command_cluster_raft_ps.go

chore: execute goimports to format the code (#7983 )

2026-01-07 13:06:08 -08:00

command_cluster_raft_remove.go

chore: execute goimports to format the code (#7983 )

2026-01-07 13:06:08 -08:00

command_cluster_status_test.go

Update cluster.status to resolve file details on EC volumes. (#8268 )

2026-02-09 17:52:43 -08:00

command_cluster_status.go

Fix file stat collection metric bug for the cluster.status command. (#8302 )

2026-02-11 13:34:20 -08:00

command_collection_delete.go

Unify the parameter to disable dry-run on weed shell commands to -apply (instead of -force). (#7450 )

2025-11-09 19:58:38 -08:00

command_collection_list.go

chore: execute goimports to format the code (#7983 )

2026-01-07 13:06:08 -08:00

command_ec_balance.go

add admin script worker (#8491 )

2026-03-03 15:10:40 -08:00

command_ec_common_avoid_test.go

Enhance EC balancing to separate parity and data shards (#8038 )

2026-01-15 12:43:44 -08:00

command_ec_common_test.go

fix: EC rebalance fails with replica placement 000 (#7812 )

2025-12-19 13:29:12 -08:00

command_ec_common.go

add admin script worker (#8491 )

2026-03-03 15:10:40 -08:00

command_ec_decode.go

Respect -minFreeSpace during ec.decode (#8467 )

2026-02-27 23:54:30 -08:00

command_ec_encode_test.go

add tests

2026-02-09 01:37:56 -08:00

command_ec_encode.go

fix ec.encode skipping volumes when one replica is on a full disk (#8227 )

2026-02-09 14:23:11 -08:00

command_ec_rebuild_test.go

fix(ec): gather shards from all disk locations before rebuild (#8633 )

2026-03-14 20:59:47 -07:00

command_ec_rebuild.go

Fix ec.rebuild failing on unrepairable volumes instead of skipping (#8632 )

2026-03-14 16:18:29 -07:00

command_ec_scrub.go

Implement local scrubbing for EC volumes. (#8283 )

2026-02-11 11:04:08 -08:00

command_ec_test.go

Fix reporting of EC shard sizes from nodes to masters. (#7835 )

2025-12-28 19:30:42 -08:00

command_fs_cat.go

Shell: Added a helper function isHelpRequest() (#7380 )

2025-10-24 20:21:35 -07:00

command_fs_cd.go

Shell: Added a helper function isHelpRequest() (#7380 )

2025-10-24 20:21:35 -07:00

command_fs_configure.go

Unify the parameter to disable dry-run on weed shell commands to -apply (instead of -force). (#7450 )

2025-11-09 19:58:38 -08:00

command_fs_du.go

Shell: Added a helper function isHelpRequest() (#7380 )

2025-10-24 20:21:35 -07:00

command_fs_log.go

chore: execute goimports to format the code (#7983 )

2026-01-07 13:06:08 -08:00

command_fs_ls.go

Shell: Added a helper function isHelpRequest() (#7380 )

2025-10-24 20:21:35 -07:00

command_fs_merge_volumes.go

fix(shell): show planned size in fs.mergeVolumes log to clarify size limit check (#8553 )

2026-03-11 13:56:13 -07:00

command_fs_meta_cat.go

Shell: Added a helper function isHelpRequest() (#7380 )

2025-10-24 20:21:35 -07:00

command_fs_meta_change_volume_id.go

Fix volume.fsck -forcePurging -reallyDeleteFromVolume to fail fast on filer traversal errors (#8015 )

2026-01-14 21:37:50 -08:00

command_fs_meta_load.go

use ReadFull (#40 ) (#8240 )

2026-02-06 20:51:47 -08:00

command_fs_meta_notify.go

Fix volume.fsck -forcePurging -reallyDeleteFromVolume to fail fast on filer traversal errors (#8015 )

2026-01-14 21:37:50 -08:00

command_fs_meta_save.go

shell: fix potential deadlock in fs.meta.save BFS traversal

2026-01-29 14:42:10 -08:00

command_fs_mkdir.go

Shell: Added a helper function isHelpRequest() (#7380 )

2025-10-24 20:21:35 -07:00

command_fs_mv.go

Shell: Added a helper function isHelpRequest() (#7380 )

2025-10-24 20:21:35 -07:00

command_fs_pwd.go

Shell: Added a helper function isHelpRequest() (#7380 )

2025-10-24 20:21:35 -07:00

command_fs_rm.go

Shell: Added a helper function isHelpRequest() (#7380 )

2025-10-24 20:21:35 -07:00

command_fs_tree.go

Shell: Added a helper function isHelpRequest() (#7380 )

2025-10-24 20:21:35 -07:00

command_fs_verify.go

shell: update fs.verify and volume.fsck for new BFS signature

2026-01-29 14:42:10 -08:00

command_lock_unlock.go

chore: execute goimports to format the code (#7983 )

2026-01-07 13:06:08 -08:00

command_mount_configure.go

Clean up logs and deprecated functions (#7339 )

2025-10-17 22:11:50 -07:00

command_mq_balance.go

chore: execute goimports to format the code (#7983 )

2026-01-07 13:06:08 -08:00

command_mq_topic_compact.go

Add Kafka Gateway (#7231 )

2025-10-13 18:05:17 -07:00

command_mq_topic_configure.go

chore: execute goimports to format the code (#7983 )

2026-01-07 13:06:08 -08:00

command_mq_topic_desc.go

chore: execute goimports to format the code (#7983 )

2026-01-07 13:06:08 -08:00

command_mq_topic_list.go

chore: execute goimports to format the code (#7983 )

2026-01-07 13:06:08 -08:00

command_mq_topic_truncate.go

Message Queue: Add sql querying (#7185 )

2025-09-09 01:01:03 -07:00

command_remote_cache.go

improve: large file sync throughput for remote.cache and filer.sync (#8676 )

2026-03-17 16:49:56 -07:00

command_remote_configure.go

add one example

2025-12-28 11:39:06 -08:00

command_remote_copy_local.go

fix(shell): set LastLocalSyncTsNs in remote.copy.local so remote.uncache works (#8604 )

2026-03-11 12:55:45 -07:00

command_remote_meta_sync.go

Fix remote.meta.sync TTL issue (#8021 ) (#8030 )

2026-01-14 14:45:52 -08:00

command_remote_mount_buckets.go

feat(remote.mount): add -metadataStrategy flag to control metadata caching (#8568 )

2026-03-12 15:21:07 -07:00

command_remote_mount.go

feat(filer): add lazy directory listing for remote mounts (#8615 )

2026-03-13 09:36:54 -07:00

command_remote_uncache_test.go

shell: add minCacheAge flag to remote.uncache command (#8225 )

2026-02-05 12:57:27 -08:00

command_remote_uncache.go

shell: add minCacheAge flag to remote.uncache command (#8225 )

2026-02-05 12:57:27 -08:00

command_remote_unmount.go

chore: execute goimports to format the code (#7983 )

2026-01-07 13:06:08 -08:00

command_s3_bucket_create.go

feat(shell): add Object Lock management commands (#8141 )

2026-01-27 10:50:16 -08:00

command_s3_bucket_delete.go

fix: admin UI bucket delete now properly deletes collection and checks Object Lock (#7734 )

2025-12-13 13:41:25 -08:00

command_s3_bucket_list.go

shell: add -owner flag to s3.bucket.create command (#7728 )

2025-12-12 18:06:13 -08:00

command_s3_bucket_lock.go

feat(shell): add Object Lock management commands (#8141 )

2026-01-27 10:50:16 -08:00

command_s3_bucket_owner.go

shell: add -owner flag to s3.bucket.create command (#7728 )

2025-12-12 18:06:13 -08:00

command_s3_bucket_quota_check.go

Reduce memory allocations in hot paths (#7725 )

2025-12-12 12:51:48 -08:00

command_s3_bucket_quota.go

convert error fromating to %w everywhere (#6995 )

2025-07-16 23:39:27 -07:00

command_s3_bucket_versioning.go

Chart createBuckets config #8368 : Add TTL, Object Lock, and Versioning support (#8375 )

2026-02-26 11:56:10 -08:00

command_s3_circuitbreaker_test.go

refactor(shell): readability improvements (#3704 )

2022-09-16 02:43:49 -07:00

command_s3_circuitbreaker.go

chore(deps): bump gocloud.dev from 0.40.0 to 0.41.0 (#6679 )

2025-03-31 21:42:54 -07:00

command_s3_clean_uploads.go

convert error fromating to %w everywhere (#6995 )

2025-07-16 23:39:27 -07:00

command_s3_configure.go

fix(iam): ensure access key status is persisted and defaulted to Active (#8341 )

2026-02-13 20:28:41 -08:00

command_s3_policy.go

IAM Policy Management via gRPC (#8109 )

2026-01-25 13:39:30 -08:00

command_s3tables_bucket.go

Add s3tables shell and admin UI (#8172 )

2026-01-30 22:57:05 -08:00

command_s3tables_namespace.go

Add s3tables shell and admin UI (#8172 )

2026-01-30 22:57:05 -08:00

command_s3tables_table.go

Add s3tables shell and admin UI (#8172 )

2026-01-30 22:57:05 -08:00

command_s3tables_tag.go

Add s3tables shell and admin UI (#8172 )

2026-01-30 22:57:05 -08:00

command_sleep.go

add admin script worker (#8491 )

2026-03-03 15:10:40 -08:00

command_volume_balance_test.go

Shell: support regular expression for collection selection (#7158 )

2025-08-23 11:04:24 -07:00

command_volume_balance.go

[shell]: volume balance capacity by min volume density (#8026 )

2026-02-19 13:30:59 -08:00

command_volume_check_disk_test.go

Mutex command output writes for volume.check.disk. (#7605 )

2025-12-02 10:14:24 -08:00

command_volume_check_disk.go

Fix handling of fixed read-only volumes for volume.check.disk. (#7612 )

2025-12-03 11:33:35 -08:00

command_volume_configure_replication.go

Fix #8040 : Support '_default' keyword in collectionPattern to match default collection (#8046 )

2026-01-16 12:31:48 -08:00

command_volume_copy.go

Fix live volume move tail timestamp (#8440 )

2026-02-24 20:07:26 -08:00

command_volume_delete_empty.go

Unify the parameter to disable dry-run on weed shell commands to -apply (instead of -force). (#7450 )

2025-11-09 19:58:38 -08:00

command_volume_delete.go

chore: execute goimports to format the code (#7983 )

2026-01-07 13:06:08 -08:00

command_volume_fix_replication_test.go

Fix/copy before delete replication (#6064 )

2024-09-26 08:34:13 -07:00

command_volume_fix_replication.go

Fix #8040 : Support '_default' keyword in collectionPattern to match default collection (#8046 )

2026-01-16 12:31:48 -08:00

command_volume_fsck.go

Fix volume.fsck crashing on EC volumes and add multi-volume vacuum support (#8406 )

2026-02-22 22:07:15 -08:00

command_volume_grow.go

chore: execute goimports to format the code (#7983 )

2026-01-07 13:06:08 -08:00

command_volume_list_test.go

opt: reduce ShardsInfo memory usage with bitmap and sorted slice (#7974 )

2026-01-06 00:09:52 -08:00

command_volume_list.go

Fix inconsistent TTL reporting in volume.list #8158 (#8164 )

2026-01-29 14:16:42 -08:00

command_volume_mark.go

refactor

2024-09-29 10:38:22 -07:00

command_volume_merge_test.go

go fmt

2026-02-25 10:25:23 -08:00

command_volume_merge.go

Adds volume.merge command with deduplication and disk-based backend (#8441 )

2026-02-25 10:12:09 -08:00

command_volume_mount.go

chore: execute goimports to format the code (#7983 )

2026-01-07 13:06:08 -08:00

command_volume_move.go

Fix live volume move tail timestamp (#8440 )

2026-02-24 20:07:26 -08:00

command_volume_replica_check_test.go

chore: execute goimports to format the code (#7983 )

2026-01-07 13:06:08 -08:00

command_volume_replica_check.go

fix: sync replica entries before ec.encode and volume.tier.move (#7798 )

2025-12-16 23:16:07 -08:00

command_volume_scrub.go

Implement full scrubbing for regular volumes (#8254 )

2026-02-13 15:47:29 -08:00

command_volume_server_evacuate_test.go

fix tests

2023-03-13 10:00:27 -07:00

command_volume_server_evacuate.go

Enhance EC balancing to separate parity and data shards (#8038 )

2026-01-15 12:43:44 -08:00

command_volume_server_leave.go

Unify the parameter to disable dry-run on weed shell commands to -apply (instead of -force). (#7450 )

2025-11-09 19:58:38 -08:00

command_volume_server_state.go

Add weed shell command volumeServer.state to query/update volume server state settings. (#8271 )

2026-02-11 11:02:37 -08:00

command_volume_tier_compact_test.go

feat(shell): add volume.tier.compact command to reclaim cloud storage space (#8715 )

2026-03-20 23:52:12 -07:00

command_volume_tier_compact.go

feat(shell): add volume.tier.compact command to reclaim cloud storage space (#8715 )

2026-03-20 23:52:12 -07:00

command_volume_tier_download.go

Shell: support regular expression for collection selection (#7158 )

2025-08-23 11:04:24 -07:00

command_volume_tier_move.go

Fix #8040 : Support '_default' keyword in collectionPattern to match default collection (#8046 )

2026-01-16 12:31:48 -08:00

command_volume_tier_upload.go

Shell: support regular expression for collection selection (#7158 )

2025-08-23 11:04:24 -07:00

command_volume_unmount.go

chore: execute goimports to format the code (#7983 )

2026-01-07 13:06:08 -08:00

command_volume_vacuum_disable.go

refactor

2024-09-29 10:38:22 -07:00

command_volume_vacuum_enable.go

refactor

2024-09-29 10:38:22 -07:00

command_volume_vacuum.go

Fix volume.fsck crashing on EC volumes and add multi-volume vacuum support (#8406 )

2026-02-22 22:07:15 -08:00

command.go

refactor

2024-09-29 10:38:22 -07:00

commands.go

add admin script worker (#8491 )

2026-03-03 15:10:40 -08:00

common_test.go

Humanize output for weed.server by default (#7758 )

2025-12-15 11:18:45 -08:00

common.go

Fix #8040 : Support '_default' keyword in collectionPattern to match default collection (#8046 )

2026-01-16 12:31:48 -08:00

ec_proportional_rebalance_test.go

fix: EC rebalance fails with replica placement 000 (#7812 )

2025-12-19 13:29:12 -08:00

ec_proportional_rebalance.go

Fix reporting of EC shard sizes from nodes to masters. (#7835 )

2025-12-28 19:30:42 -08:00

ec_rebalance_slots_test.go

opt: reduce ShardsInfo memory usage with bitmap and sorted slice (#7974 )

2026-01-06 00:09:52 -08:00

s3tables_helpers.go

Add s3tables shell and admin UI (#8172 )

2026-01-30 22:57:05 -08:00

shell_liner_test.go

shell: allow spaces in arguments via quoting (#8157 ) (#8165 )

2026-01-29 19:06:17 -08:00

shell_liner.go

shell: allow spaces in arguments via quoting (#8157 ) (#8165 )

2026-01-29 19:06:17 -08:00

volume.ecshards.txt

Humanize output for weed.server by default (#7758 )

2025-12-15 11:18:45 -08:00

volume.list2.txt

Humanize output for weed.server by default (#7758 )

2025-12-15 11:18:45 -08:00

volume.list.txt

Humanize output for weed.server by default (#7758 )

2025-12-15 11:18:45 -08:00