Commit Graph

11469 Commits

Author SHA1 Message Date
chrislu
f3d44b1eb6 Update fuse.go
Changed the signal from SIGUSR1 to SIGTERM. This should fix the compilation error since SIGTERM is available on all Unix-like systems including macOS. The functionality remains similar - it will still allow the master process to wait for a signal before exiting, just using a different signal type.
2025-05-22 11:32:08 -07:00
chrislu
896096c41b fix compilation on windows 2025-05-22 11:25:56 -07:00
chrislu
25d6310307 fix error format 2025-05-22 09:56:31 -07:00
Weihao Jiang
874b4a5535 Ensure weed fuse master process exits after mounted (#6809)
* Ensure fuse master process wait for mounted

* Validate parent PID input in fuse command
2025-05-22 09:50:07 -07:00
Aleksey Kosov
165af32d6b added context to filer_client method calls (#6808)
Co-authored-by: akosov <a.kosov@kryptonite.ru>
2025-05-22 09:46:49 -07:00
Aleksey Kosov
5182d46e22 Added middleware for processing request_id grpc and http requests (#6805) 2025-05-21 07:57:39 -07:00
dependabot[bot]
140b7a7402 chore(deps): bump docker/build-push-action from 6.16.0 to 6.17.0 (#6796)
Bumps [docker/build-push-action](https://github.com/docker/build-push-action) from 6.16.0 to 6.17.0.
- [Release notes](https://github.com/docker/build-push-action/releases)
- [Commits](14487ce63c...1dc7386353)

---
updated-dependencies:
- dependency-name: docker/build-push-action
  dependency-version: 6.17.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-05-19 11:18:50 -07:00
dependabot[bot]
c25e455111 chore(deps): bump actions/dependency-review-action from 4.7.0 to 4.7.1 (#6797)
Bumps [actions/dependency-review-action](https://github.com/actions/dependency-review-action) from 4.7.0 to 4.7.1.
- [Release notes](https://github.com/actions/dependency-review-action/releases)
- [Commits](38ecb5b593...da24556b54)

---
updated-dependencies:
- dependency-name: actions/dependency-review-action
  dependency-version: 4.7.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-05-19 11:18:40 -07:00
dependabot[bot]
01306b66ba chore(deps): bump github.com/tarantool/go-tarantool/v2 from 2.3.0 to 2.3.2 (#6798)
chore(deps): bump github.com/tarantool/go-tarantool/v2

Bumps [github.com/tarantool/go-tarantool/v2](https://github.com/tarantool/go-tarantool) from 2.3.0 to 2.3.2.
- [Release notes](https://github.com/tarantool/go-tarantool/releases)
- [Changelog](https://github.com/tarantool/go-tarantool/blob/master/CHANGELOG.md)
- [Commits](https://github.com/tarantool/go-tarantool/compare/v2.3.0...v2.3.2)

---
updated-dependencies:
- dependency-name: github.com/tarantool/go-tarantool/v2
  dependency-version: 2.3.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-05-19 11:18:29 -07:00
dependabot[bot]
31660aa8e8 chore(deps): bump github.com/aws/aws-sdk-go from 1.55.6 to 1.55.7 (#6799)
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.55.6 to 1.55.7.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Changelog](https://github.com/aws/aws-sdk-go/blob/main/CHANGELOG_PENDING.md)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.55.6...v1.55.7)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
  dependency-version: 1.55.7
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-05-19 11:18:18 -07:00
dependabot[bot]
0cc965737b chore(deps): bump go.etcd.io/etcd/client/v3 from 3.5.18 to 3.6.0 (#6800)
Bumps [go.etcd.io/etcd/client/v3](https://github.com/etcd-io/etcd) from 3.5.18 to 3.6.0.
- [Release notes](https://github.com/etcd-io/etcd/releases)
- [Commits](https://github.com/etcd-io/etcd/compare/v3.5.18...v3.6.0)

---
updated-dependencies:
- dependency-name: go.etcd.io/etcd/client/v3
  dependency-version: 3.6.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-05-19 11:17:37 -07:00
dependabot[bot]
6fd752445e chore(deps): bump cloud.google.com/go/storage from 1.51.0 to 1.54.0 (#6801)
Bumps [cloud.google.com/go/storage](https://github.com/googleapis/google-cloud-go) from 1.51.0 to 1.54.0.
- [Release notes](https://github.com/googleapis/google-cloud-go/releases)
- [Changelog](https://github.com/googleapis/google-cloud-go/blob/main/CHANGES.md)
- [Commits](https://github.com/googleapis/google-cloud-go/compare/spanner/v1.51.0...spanner/v1.54.0)

---
updated-dependencies:
- dependency-name: cloud.google.com/go/storage
  dependency-version: 1.54.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-05-19 11:16:55 -07:00
Piotr Sawicki
372547e332 Helm chart - volume statefulset - volumeClaimTemplates - add apiVersion and kind for better compatibility with argocd (#6793) 2025-05-16 00:33:57 -07:00
dongxufeng
ff878a542d correctly report volume with input/output error to master (#6790)
* correctly capture io error and report to master

* code fix

* check io error by error.Is

---------

Co-authored-by: dongxu_feng <dongxu_feng@intsig.net>
2025-05-15 00:56:43 -07:00
chrislu
d8c574a5ef fix fsync logic 2025-05-14 01:33:36 -07:00
Lisandro Pin
0be020b0fa Nit: unify the default --maxParallelization value for weed shell commands supporting this option (#6788) 2025-05-13 07:59:26 -07:00
Lisandro Pin
ba1d82db90 Move shell.ErrorWaitGroup into a common file, to cleanly reuse across weed shell commands. (#6780)
Move `shell.ErrorWaitGroup` into a dedicated common file, to cleanly reuse across `weed shell` commands.
2025-05-12 14:38:55 -07:00
dependabot[bot]
b0f7c33ce0 chore(deps): bump github.com/redis/go-redis/v9 from 9.7.3 to 9.8.0 (#6781)
Bumps [github.com/redis/go-redis/v9](https://github.com/redis/go-redis) from 9.7.3 to 9.8.0.
- [Release notes](https://github.com/redis/go-redis/releases)
- [Changelog](https://github.com/redis/go-redis/blob/master/CHANGELOG.md)
- [Commits](https://github.com/redis/go-redis/compare/v9.7.3...v9.8.0)

---
updated-dependencies:
- dependency-name: github.com/redis/go-redis/v9
  dependency-version: 9.8.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-05-12 14:38:23 -07:00
dependabot[bot]
c993a43a91 chore(deps): bump golang.org/x/tools from 0.31.0 to 0.33.0 (#6782)
Bumps [golang.org/x/tools](https://github.com/golang/tools) from 0.31.0 to 0.33.0.
- [Release notes](https://github.com/golang/tools/releases)
- [Commits](https://github.com/golang/tools/compare/v0.31.0...v0.33.0)

---
updated-dependencies:
- dependency-name: golang.org/x/tools
  dependency-version: 0.33.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-05-12 14:38:15 -07:00
dependabot[bot]
6eaa35040b chore(deps): bump github.com/ydb-platform/ydb-go-sdk/v3 from 3.106.1 to 3.108.3 (#6783)
chore(deps): bump github.com/ydb-platform/ydb-go-sdk/v3

Bumps [github.com/ydb-platform/ydb-go-sdk/v3](https://github.com/ydb-platform/ydb-go-sdk) from 3.106.1 to 3.108.3.
- [Release notes](https://github.com/ydb-platform/ydb-go-sdk/releases)
- [Changelog](https://github.com/ydb-platform/ydb-go-sdk/blob/master/CHANGELOG.md)
- [Commits](https://github.com/ydb-platform/ydb-go-sdk/compare/v3.106.1...v3.108.3)

---
updated-dependencies:
- dependency-name: github.com/ydb-platform/ydb-go-sdk/v3
  dependency-version: 3.108.3
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-05-12 14:38:03 -07:00
dependabot[bot]
8da3079d22 chore(deps): bump actions/dependency-review-action from 4.6.0 to 4.7.0 (#6784)
Bumps [actions/dependency-review-action](https://github.com/actions/dependency-review-action) from 4.6.0 to 4.7.0.
- [Release notes](https://github.com/actions/dependency-review-action/releases)
- [Commits](ce3cf9537a...38ecb5b593)

---
updated-dependencies:
- dependency-name: actions/dependency-review-action
  dependency-version: 4.7.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-05-12 14:37:52 -07:00
SmoothDenis
45964c2f86 fix: ydb filer bugs (#6778)
* fix: ydb filer bugs

* fix(ydb): correct DeleteEntry log argument types

* fix(ydb): bucket creation & deletion logic
2025-05-12 01:20:25 -07:00
Kuzmin Anton
9c1048bacb Add prefix listing in mongodb_store.go (#6777) 2025-05-11 07:13:49 -07:00
Lisandro Pin
dddb0f0ae5 Fix update of SeaweedFS_volumeServer_volumes gauge metrics when EC shards are unmounted (#6776) 2025-05-09 10:15:34 -07:00
orthoxerox
d8cc269294 feature: added ssl support for HCFS (#6699) (#6775) 2025-05-09 09:09:13 -07:00
Lisandro Pin
848d1f7c34 Improve safety for weed shell's ec.encode. (#6773)
Improve safety for weed shells `ec.encode`.

The current process for `ec.encode` is:

1. EC shards for a volume are generated and added to a single server
2. The original volume is deleted
3. EC shards get re-balanced across the entire topology

It is then possible to lose data between #2 and #3, if the underlying volume storage/server/rack/DC
happens to fail, for whatever reason. As a fix, this MR reworks `ec.encode` so:

  * Newly created EC shards are spread across all locations for the source volume.
  * Source volumes are deleted only after EC shards are converted and balanced.
2025-05-09 09:01:32 -07:00
Quentin D.
2ae5b480a6 Use the correct constant when computing the offset in SearchNeedleFromSortedIndex (#6771)
NeedleHeaderSize happen to have the same size as NeedleMapEntrySize, except when running the 5 bytes offset variant of Seaweedfs, because it does not contain OffsetSize. This causes ECX corruption on deletes, due to the drifting offset computation (offset is always computed on a basis of 16 bytes per record instead of 17 bytes)

Signed-off-by: Quentin Devos <4972091+Okhoshi@users.noreply.github.com>
2025-05-09 08:47:53 -07:00
Lisandro Pin
97dad06ed8 Improve parallelization for ec.encode (#6769)
Improve parallelization for `ec.encode`.

Instead of processing one volume at at time, perform all EC conversion
steps (mark readonly -> generate EC shards -> delete volume -> remount) in
parallel for all of them.

This should substantially improve performance when EC encoding
entire collections.
2025-05-08 17:14:14 -07:00
Bruce Zou
582206b7d2 load configuration before get max_file_name_length (#6766) 2025-05-07 21:52:36 -07:00
dependabot[bot]
7522f11fd7 chore(deps): bump github.com/go-sql-driver/mysql from 1.9.1 to 1.9.2 (#6720)
Bumps [github.com/go-sql-driver/mysql](https://github.com/go-sql-driver/mysql) from 1.9.1 to 1.9.2.
- [Release notes](https://github.com/go-sql-driver/mysql/releases)
- [Changelog](https://github.com/go-sql-driver/mysql/blob/master/CHANGELOG.md)
- [Commits](https://github.com/go-sql-driver/mysql/compare/v1.9.1...v1.9.2)

---
updated-dependencies:
- dependency-name: github.com/go-sql-driver/mysql
  dependency-version: 1.9.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-05-05 11:55:21 -07:00
Manuel Leonhardt
3c15c8ddad fix: remove blocking match label (#6760)
matchLabels are immutable. Thus, matching against the version, which
changes with every release, blocks Helm upgrades. To resolve this, the
label is removed. See [1] where this was similarly done.

[1] https://github.com/seaweedfs/seaweedfs/pull/6090
2025-05-05 11:54:43 -07:00
Mohamed Sekour
93aed187e9 Add SFTP Server Support (#6753)
* Add SFTP Server Support

Signed-off-by: Mohamed Sekour <mohamed.sekour@exfo.com>

* fix s3 tests and helm lint

Signed-off-by: Mohamed Sekour <mohamed.sekour@exfo.com>

* increase  helm chart version

* adjust version

---------

Signed-off-by: Mohamed Sekour <mohamed.sekour@exfo.com>
Co-authored-by: chrislu <chris.lu@gmail.com>
2025-05-05 11:43:49 -07:00
ARibster
a2c5510ae1 Addendum to issue #6733 (#6756)
add s3.ip.bind command line parameter to filer
2025-05-05 10:30:54 -07:00
dependabot[bot]
a2473af05a chore(deps): bump github.com/aws/aws-sdk-go-v2/config from 1.29.13 to 1.29.14 (#6757)
chore(deps): bump github.com/aws/aws-sdk-go-v2/config

Bumps [github.com/aws/aws-sdk-go-v2/config](https://github.com/aws/aws-sdk-go-v2) from 1.29.13 to 1.29.14.
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/main/changelog-template.json)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/config/v1.29.13...config/v1.29.14)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go-v2/config
  dependency-version: 1.29.14
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-05-05 10:30:31 -07:00
dependabot[bot]
51b0f9b7fd chore(deps): bump github.com/prometheus/procfs from 0.16.0 to 0.16.1 (#6758)
Bumps [github.com/prometheus/procfs](https://github.com/prometheus/procfs) from 0.16.0 to 0.16.1.
- [Release notes](https://github.com/prometheus/procfs/releases)
- [Commits](https://github.com/prometheus/procfs/compare/v0.16.0...v0.16.1)

---
updated-dependencies:
- dependency-name: github.com/prometheus/procfs
  dependency-version: 0.16.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-05-05 10:30:21 -07:00
dependabot[bot]
5aff086aa4 chore(deps): bump actions/dependency-review-action from 4.3.2 to 4.6.0 (#6759)
Bumps [actions/dependency-review-action](https://github.com/actions/dependency-review-action) from 4.3.2 to 4.6.0.
- [Release notes](https://github.com/actions/dependency-review-action/releases)
- [Commits](0c155c5e85...ce3cf9537a)

---
updated-dependencies:
- dependency-name: actions/dependency-review-action
  dependency-version: 4.6.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-05-05 10:30:12 -07:00
chrislu
e4e8505208 3.87 2025-05-04 21:10:01 -07:00
chrislu
ed15357c7a adds tls examples
related to https://github.com/seaweedfs/seaweedfs/pull/6738
2025-05-04 05:46:33 -07:00
Jade Devin Cabatlao
21514e1fea feat(redis): add mTLS support for Redis connection initialization (#6738)
* feat(redis): add mTLS support for Redis connection initialization

- Enhanced the Redis2Store initialization to support mutual TLS (mTLS) by adding configuration options for CA certificate, client certificate, and client key paths.
- Updated the Redis client setup to use TLS configuration when mTLS is enabled, ensuring secure connections to the Redis server.

* feat(redis): extend Redis3Store initialization to support mTLS

- Added configuration options for enabling mutual TLS (mTLS) in Redis3Store.
- Implemented logic to load client certificates and CA certificates for secure Redis connections.
- Updated the Redis client setup to utilize TLS configuration when mTLS is enabled.

---------

Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
2025-05-04 05:31:27 -07:00
dependabot[bot]
4e7948abd2 chore(deps): bump github.com/go-jose/go-jose/v4 from 4.0.4 to 4.0.5 (#6750)
Bumps [github.com/go-jose/go-jose/v4](https://github.com/go-jose/go-jose) from 4.0.4 to 4.0.5.
- [Release notes](https://github.com/go-jose/go-jose/releases)
- [Changelog](https://github.com/go-jose/go-jose/blob/main/CHANGELOG.md)
- [Commits](https://github.com/go-jose/go-jose/compare/v4.0.4...v4.0.5)

---
updated-dependencies:
- dependency-name: github.com/go-jose/go-jose/v4
  dependency-version: 4.0.5
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-05-01 18:25:34 -07:00
dependabot[bot]
b45f43ded1 chore(deps): bump google.golang.org/api from 0.228.0 to 0.230.0 (#6742)
Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client) from 0.228.0 to 0.230.0.
- [Release notes](https://github.com/googleapis/google-api-go-client/releases)
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md)
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.228.0...v0.230.0)

---
updated-dependencies:
- dependency-name: google.golang.org/api
  dependency-version: 0.230.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-05-01 13:10:14 -07:00
dependabot[bot]
f14de131b3 chore(deps): bump docker/build-push-action from 6.15.0 to 6.16.0 (#6743)
Bumps [docker/build-push-action](https://github.com/docker/build-push-action) from 6.15.0 to 6.16.0.
- [Release notes](https://github.com/docker/build-push-action/releases)
- [Commits](471d1dc4e0...14487ce63c)

---
updated-dependencies:
- dependency-name: docker/build-push-action
  dependency-version: 6.16.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-05-01 13:10:02 -07:00
ARibster
d424cbbcd5 fix: S3 Feature: please add s3.idleTimeout command line parameter #6746 (#6747)
ildeTimeout command line parameter for s3
2025-05-01 13:09:47 -07:00
Taehyung Lim
71382d3279 bump depdency-review-action 4.3.0 -> 4.3.2 (#6732) 2025-05-01 13:08:45 -07:00
Taehyung Lim
0fbdedecc7 [s3] refactor duplicate put-object code paths (#6727) 2025-05-01 13:08:03 -07:00
Taehyung Lim
affacb1b67 [s3] add tests for s3 over https using aws-cli (#6731)
* [s3] add tests for s3 over https using aws-cli

* fix typo
2025-05-01 13:06:59 -07:00
ARibster
9f4be83e88 fix #6733 add s3.ip.bind command line parameter (#6741) 2025-04-28 04:49:01 -07:00
NyaMisty
8d0e6f1ead fix: volume.list volume info output not in order (#6737) 2025-04-27 08:52:49 -07:00
Konstantin Lebedev
c8b0a04409 golang up version to 1.24 (#6724) 2025-04-23 15:49:28 -07:00
Taehyung Lim
ea941462f9 [s3] use chunkedReader in PutObjectPartHandler to handle aws-chunked encoding (#6725) 2025-04-23 15:45:46 -07:00