Commit Graph

93 Commits

Author SHA1 Message Date
Chris Lu
0a46577700 Fix #8040: Support '_default' keyword in collectionPattern to match default collection (#8046)
* Fix #8040: Support 'default' keyword in collectionPattern to match default collection

The default collection in SeaweedFS is represented as an empty string internally.
Previously, it was impossible to specifically target only the default collection
because:
- Empty collectionPattern matched ALL collections (filter was skipped)
- Using collectionPattern="default" tried to match the literal string "default"

This commit adds special handling for the keyword "default" in collectionPattern
across multiple shell commands:
- volume.tier.move
- volume.list
- volume.fix.replication
- volume.configure.replication

Now users can use -collectionPattern="default" to specifically target volumes
in the default collection (empty collection name), while maintaining backward
compatibility where empty pattern matches all collections.

Updated help text to document this feature.

* Update compileCollectionPattern to support 'default' keyword

This extends the fix to all commands that use regex-based collection
pattern matching:
- ec.encode
- ec.decode
- volume.tier.download
- volume.balance

The compileCollectionPattern function now treats "default" as a special
keyword that compiles to the regex "^$" (matching empty strings), making
it consistent with the other commands that use filepath.Match.

* Use CollectionDefault constant instead of hardcoded "default" string

Refactored the collection pattern matching logic to use a central constant
CollectionDefault defined in weed/shell/common.go. This improves maintainability
and ensures consistency across all shell commands.

* Address PR review feedback: simplify logic and use '_default' keyword

Changes:
1. Changed CollectionDefault from "default" to "_default" to avoid collision
   with literal collection names
2. Simplified pattern matching logic to reduce code duplication across all
   affected commands
3. Fixed error handling in command_volume_tier_move.go to properly propagate
   filepath.Match errors instead of swallowing them
4. Updated documentation to clarify how to match a literal "default"
   collection using regex patterns like "^default$"

This addresses all feedback from PR review comments.

* Remove unnecessary documentation about matching literal 'default'

Since we changed the keyword to '_default', users can now simply use
'default' to match a literal collection named "default". The previous
documentation about using regex patterns was confusing and no longer needed.

* Fix error propagation and empty pattern handling

1. command_volume_tier_move.go: Added early termination check after
   eachDataNode callback to stop processing remaining nodes if a pattern
   matching error occurred, improving efficiency

2. command_volume_configure_replication.go: Fixed empty pattern handling
   to match all collections (collectionMatched = true when pattern is empty),
   mirroring the behavior in other commands

These changes address the remaining PR review feedback.
2026-01-16 12:31:48 -08:00
Chris Lu
2b3ff3cd05 verbose mode 2025-12-26 12:42:00 -08:00
steve.wei
5c25df20f2 feat(volume.fix): show all replica locations for misplaced volumes (#7560) 2025-11-27 00:04:45 -08:00
Lisandro Pin
9744382a18 Rework parameters passing for functions within volume.check.disk. (#7448)
* Rework parameters passing for functions within `volume.check.disk`.

We'll need to rework this logic to account for read-only volumes, and there're already way too many parameters shuffled around.

Grouping these into a single struct simplifies the overall codebase.

* similar fix

* Improved Error Handling in Tests

* propagate the errors

* edge cases

* edge case on modified time

* clean up

---------

Co-authored-by: chrislu <chris.lu@gmail.com>
2025-11-10 16:03:38 -08:00
Lisandro Pin
76e4a51964 Unify the parameter to disable dry-run on weed shell commands to -apply (instead of -force). (#7450)
* Unify the parameter to disable dry-run on weed shell commands to --apply (instead of --force).

* lint

* refactor

* Execution Order Corrected

* handle deprecated force flag

* fix help messages

* Refactoring]: Using flag.FlagSet.Visit()

* consistent with other commands

* Checks for both flags

* fix toml files

---------

Co-authored-by: chrislu <chris.lu@gmail.com>
2025-11-09 19:58:38 -08:00
nightcoffee
aed91baa2e [weed] update volume.fix.replication description (#7340)
* [weed] update volume.fix.replication description

* Update master-cloud.toml

* Update master.toml
2025-10-21 12:38:40 -07:00
Roman Shishkin
b6f5fb4b45 Human-readable processed bytes in volume.fix.replication (#7253) 2025-09-18 06:44:40 -07:00
Lisandro Pin
64198dad83 Paralleize operations for weed shell's volume.fix.replication. (#6789)
Paralleize operations for `weed shell`s `volume.fix.replication`.
2025-07-30 10:58:30 -07:00
dsd
da2a234b00 [weed] change -n to -force (#6421) 2025-01-08 09:57:18 -08:00
dsd
20cbc9e4eb skip error while executing volume.fix.replication (#6382) 2024-12-20 07:36:13 -08:00
chrislu
ec155022e7 "golang.org/x/exp/slices" => "slices" and go fmt 2024-12-19 19:25:06 -08:00
Trim21
d43fa07f06 use readable bytes size string in shell output (#6288) 2024-11-25 17:25:17 -08:00
Konstantin Lebedev
a143c888e5 [shell] don't require lock when there are no changes for volume.fix.replication (#6266)
* don't require lock when there are no changes

* revert takeAction
2024-11-21 08:17:25 -08:00
chrislu
07cf8cf22d minor 2024-11-19 08:31:33 -08:00
Lisandro Pin
0d5393641e Unify usage of shell.EcNode.dc as DataCenterId. (#6258) 2024-11-19 06:33:18 -08:00
chrislu
20929f2a57 adjust resource heavy for volume.fix.replication 2024-09-29 11:32:18 -07:00
chrislu
6564ceda91 skip resource heavy commands from running on master nodes 2024-09-29 10:51:17 -07:00
chrislu
ec30a504ba refactor 2024-09-29 10:38:22 -07:00
chrislu
701abbb9df add IsResourceHeavy() to command interface 2024-09-28 20:23:01 -07:00
Max Denushev
f1e700ce2f Fix/copy before delete replication (#6064)
* fix(shell): volume.fix.replication misplaced volumes unsatisfying replication factor

* fix(shell): simplify replication check

* fix(shell): add test for satisfyReplicaCurrentLocation
2024-09-26 08:34:13 -07:00
skycope
6e4b9181f5 fix "volume.fix.replication" move many replications only to one volumeServer (#5522) 2024-04-23 06:33:50 -07:00
steve.wei
67ead9b18f fix(volume.fix.replication): adjust volume count, not free volume count (#5479) 2024-04-08 07:30:04 -07:00
zehweh
2b9dda7d2e fix isMisplaced() in command_volume_fix_replication.go (#4988) 2023-11-07 07:58:19 -08:00
Konstantin Lebedev
dffe00a822 fix: logger place msg (#4880) 2023-10-02 08:29:09 -07:00
Konstantin Lebedev
dd580190b4 fix: avoid deleting one replica without sync (#4875)
* fix: avoid deleting one replica without sync
https://github.com/seaweedfs/seaweedfs/issues/4647

* Update weed/shell/command_volume_fix_replication.go

Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>

* fix: revert this existing do option to positive

---------

Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
2023-09-27 23:12:10 -07:00
Konstantin Lebedev
df4ded758e fix: avoid deleting more than one replica (#4873)
https://github.com/seaweedfs/seaweedfs/issues/4647

Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
2023-09-26 00:20:48 -07:00
chrislu
645ae8c57b Revert "Revert "Merge branch 'master' of https://github.com/seaweedfs/seaweedfs""
This reverts commit 8cb42c39
2023-09-25 09:35:16 -07:00
chrislu
8cb42c39ad Revert "Merge branch 'master' of https://github.com/seaweedfs/seaweedfs"
This reverts commit 2e5aa06026, reversing
changes made to 4d414f54a2.
2023-09-18 16:12:50 -07:00
dependabot[bot]
a04bd4d26f Bump github.com/rclone/rclone from 1.63.1 to 1.64.0 (#4850)
* Bump github.com/rclone/rclone from 1.63.1 to 1.64.0

Bumps [github.com/rclone/rclone](https://github.com/rclone/rclone) from 1.63.1 to 1.64.0.
- [Release notes](https://github.com/rclone/rclone/releases)
- [Changelog](https://github.com/rclone/rclone/blob/master/RELEASE.md)
- [Commits](https://github.com/rclone/rclone/compare/v1.63.1...v1.64.0)

---
updated-dependencies:
- dependency-name: github.com/rclone/rclone
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* API changes

* go mod

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
Co-authored-by: chrislu <chris.lu@gmail.com>
2023-09-18 14:43:05 -07:00
chrislu
3365468d0d added an error message 2023-08-08 20:35:21 -07:00
Konstantin Lebedev
25535e9c36 Delete volume is empty (#4561)
* use onlyEmpty for deleteVolume
https://github.com/seaweedfs/seaweedfs/issues/4559

* fix IsEmpty

* fix test

---------

Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
2023-06-12 10:42:44 -07:00
chrislu
214b7cd286 volume.fix.replication: adjust the retry checking times 2023-02-22 10:47:52 -08:00
chrislu
21c0587900 go fmt 2022-09-14 23:06:44 -07:00
Brian
4e3e2b1b82 Add option in volume.fix.replication to only fix under-replication and not delete volumes (#3640) 2022-09-10 08:05:28 -07:00
chrislu
676e27c589 shell: stop long running jobs if lock is lost 2022-08-22 14:12:23 -07:00
chrislu
26dbc6c905 move to https://github.com/seaweedfs/seaweedfs 2022-07-29 00:17:28 -07:00
chrislu
f97acdd489 volume.fix.replication fix retry logic
fix https://github.com/chrislusf/seaweedfs/issues/3136
2022-06-03 08:45:29 -07:00
Konstantin Lebedev
44f53ceda6 fix collectionIsMismatch charset 2022-05-16 13:23:23 +05:00
Konstantin Lebedev
10d435f2c2 fix skip loop 2022-05-16 13:16:27 +05:00
Konstantin Lebedev
279053572c avoid delete volume replica if collection mismatch 2022-05-16 13:07:05 +05:00
justin
3551ca2fcf enhancement: replace sort.Slice with slices.SortFunc to reduce reflection 2022-04-18 10:35:43 +08:00
chrislu
f18803424a volume.balance: add delay during tight loop
fix https://github.com/chrislusf/seaweedfs/issues/2637
2022-02-08 00:53:55 -08:00
chrislu
9f9ef1340c use streaming mode for long poll grpc calls
streaming mode would create separate grpc connections for each call.
this is to ensure the long poll connections are properly closed.
2021-12-26 00:15:03 -08:00
chrislu
a2d3f89c7b add lock messages 2021-12-10 13:24:38 -08:00
chrislu
e6c026db65 volume.fix.replication: fix misplaced volumes
fix https://github.com/chrislusf/seaweedfs/issues/2416
2021-12-05 16:56:25 -08:00
Chris Lu
5435027ff0 volume copy: stream out copying progress and avoid grpc request timeout
fix https://github.com/chrislusf/seaweedfs/issues/2386
2021-10-24 02:52:56 -07:00
Chris Lu
e862b2529a refactor 2021-10-01 12:10:11 -07:00
Konstantin Lebedev
5e64b22b45 check that the topology has been updated 2021-10-01 18:51:22 +05:00
Konstantin Lebedev
2cecde89c3 rename opt volumesPerStep 2021-10-01 00:17:54 +05:00
Konstantin Lebedev
fc51ffce2b https://github.com/chrislusf/seaweedfs/issues/1846 2021-09-30 20:24:24 +05:00