Commit Graph

71 Commits

Author SHA1 Message Date
chrislu
8493871406 remove sqlite due to GOOS support for freebsd and openbsd 2025-07-06 21:50:08 -07:00
Chris Lu
aa66852304 Admin UI add maintenance menu (#6944)
* add ui for maintenance

* valid config loading. fix workers page.

* refactor

* grpc between admin and workers

* add a long-running bidirectional grpc call between admin and worker
* use the grpc call to heartbeat
* use the grpc call to communicate
* worker can remove the http client
* admin uses http port + 10000 as its default grpc port

* one task one package

* handles connection failures gracefully with exponential backoff

* grpc with insecure tls

* grpc with optional tls

* fix detecting tls

* change time config from nano seconds to seconds

* add tasks with 3 interfaces

* compiles reducing hard coded

* remove a couple of tasks

* remove hard coded references

* reduce hard coded values

* remove hard coded values

* remove hard coded from templ

* refactor maintenance package

* fix import cycle

* simplify

* simplify

* auto register

* auto register factory

* auto register task types

* self register types

* refactor

* simplify

* remove one task

* register ui

* lazy init executor factories

* use registered task types

* DefaultWorkerConfig remove hard coded task types

* remove more hard coded

* implement get maintenance task

* dynamic task configuration

* "System Settings" should only have system level settings

* adjust menu for tasks

* ensure menu not collapsed

* render job configuration well

* use templ for ui of task configuration

* fix ordering

* fix bugs

* saving duration in seconds

* use value and unit for duration

* Delete WORKER_REFACTORING_PLAN.md

* Delete maintenance.json

* Delete custom_worker_example.go

* remove address from workers

* remove old code from ec task

* remove creating collection button

* reconnect with exponential backoff

* worker use security.toml

* start admin server with tls info from security.toml

* fix "weed admin" cli description
2025-07-06 13:57:02 -07:00
Chris Lu
1db7c2b8aa Add credential storage (#6938)
* add credential store interface

* load credential.toml

* lint

* create credentialManager with explicit store type

* add type name

* InitializeCredentialManager

* remove unused functions

* fix missing import

* fix import

* fix nil configuration
2025-07-02 18:03:17 -07:00
Luna Yao
5e354a18a3 Update filer upsert sqlstring for postgresql (#6835) 2025-06-02 10:46:50 -07:00
bwlfhu
f1181f1121 Fix mysql tls enable (#6807) 2025-05-22 21:55:21 -07:00
chrislu
ed15357c7a adds tls examples
related to https://github.com/seaweedfs/seaweedfs/pull/6738
2025-05-04 05:46:33 -07:00
MaratKarimov
ba3afd1803 Tarantool filer store (#6669)
Co-authored-by: Marat Karimov <m.karimov@digitalms.ru>
2025-03-29 21:12:06 -07:00
Chris Lu
669a3917af add lua redis store to ensure atomicity (#6671) 2025-03-28 11:27:39 -07:00
chrislu
314f211260 Update filer.toml 2025-02-26 17:52:45 -08:00
chrislu
9bd4c47299 update redis support status 2025-02-26 17:48:44 -08:00
chrislu
4613186e8a not planning to redis3 2025-02-26 17:35:31 -08:00
Chris Lu
df436d342b add cassandra2 (#6582) 2025-02-26 17:29:22 -08:00
Chris Lu
cc05874d06 Add message queue agent (#6463)
* scaffold message queue agent

* adjust proto, add mq_agent

* add agent client implementation

* remove unused function

* agent publish server implementation

* adding agent
2025-01-20 22:19:27 -08:00
vadimartynov
914ff0e718 Replaced https.client.enabled with false in scaffold/security (#6320) 2024-12-05 10:39:28 -08:00
Konstantin Lebedev
b65eb2ec45 [security] reload whiteList on http seerver (#6302)
* reload whiteList

* white_list add to scaffold
2024-12-02 10:38:10 -08:00
zemul
6d3e38f439 Adjust threshold config (#6246)
* fix:mount deadlock

* fix

---------

Co-authored-by: zemul <zhouzemiao@ihuman.com>
2024-11-17 20:51:26 -08:00
chrislu
2074e7db67 fix invalid toml 2024-08-08 00:03:42 -07:00
vadimartynov
86d92a42b4 Added tls for http clients (#5766)
* Added global http client

* Added Do func for global http client

* Changed the code to use the global http client

* Fix http client in volume uploader

* Fixed pkg name

* Fixed http util funcs

* Fixed http client for bench_filer_upload

* Fixed http client for stress_filer_upload

* Fixed http client for filer_server_handlers_proxy

* Fixed http client for command_fs_merge_volumes

* Fixed http client for command_fs_merge_volumes and command_volume_fsck

* Fixed http client for s3api_server

* Added init global client for main funcs

* Rename global_client to client

* Changed:
- fixed NewHttpClient;
- added CheckIsHttpsClientEnabled func
- updated security.toml in scaffold

* Reduce the visibility of some functions in the util/http/client pkg

* Added the loadSecurityConfig function

* Use util.LoadSecurityConfiguration() in NewHttpClient func
2024-07-16 23:14:09 -07:00
vadimartynov
de60f383de Added tls for http handlers (#5764)
* Added https handler for filer

* Added example for security.toml
2024-07-11 07:53:18 -07:00
sb
a3a00d9499 Feature/mongodb security (#5602) 2024-05-18 12:19:10 -07:00
sb
3fae87632f add filer configuration options for username/password and tls (#5601) 2024-05-18 09:02:03 -07:00
Nikita Borzykh
1f3742850d Add TLS support, initial etcd connection check, fix key_prefix handling for etcd backend meta storage (#5403) 2024-03-24 08:36:21 -07:00
jerebear12
85d6d5371b Disable filer UI in configuration (#5297)
* Add filer.ui.enabled configuration property

* Add filer.expose_directory_metadata to config

* Ammend commit

* Remove ShowUI reference

* Update all routes that allow directory metadata

* Add cmd flag to server.go
2024-02-27 08:38:55 -08:00
jerebear12
06343f8976 Set allowed origins in config (#5109)
* Add a way to use a JWT in an HTTP only cookie

If a JWT is not included in the Authorization header or a query string, attempt to get a JWT from an HTTP only cookie.

* Added a way to specify allowed origins header from config

* Removed unecessary log

* Check list of domains from config or command flag

* Handle default wildcard and change name of config value to cors
2023-12-20 16:21:11 -08:00
Konstantin Lebedev
3c9bcfb864 chore: add dsn for connection to mysql (#5060)
* chore: add dsn for connection to mysql

* add comment

* new comment

* fix: validate dsn and adapt password
2023-12-20 16:20:58 -08:00
Konstantin Lebedev
1cac5d983d fix: disallow file name too long when writing a file (#4881)
* fix: disallow file name too long when writing a file

* bool LongerName to MaxFilenameLength

---------

Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
2023-10-12 14:29:55 -07:00
Mesar Hameed
a90b777ff4 Feat: etcd filer store keys should have customizable prefix (#4484)
An etcd cluster is not necessarily only dedicated to seaweedfs.
This security enhancement adds a customizable key_prefix option to the etcd filer store.
This will allow an etcd cluster administrator to limit the seaweedfs etcd user to only read/write a subset of keys under the
key_prefix, instead of all keys on the etcd cluster.
2023-05-18 23:08:56 -07:00
Mesar Hameed
65484e80ef Feat: support username/password authentication for etcd filer store s… (#4477)
Feat: support username/password authentication for etcd filer store seaweedfs/seaweedfs#4262

Co-authored-by: Mesar Hameed <mesar.hameed@gmail.com>
2023-05-15 17:45:14 -07:00
chrislu
efef6e94bf use UPSERT for postgres style databases 2023-04-04 11:52:36 -07:00
Muhammad Hallaj bin Subery
9bd422d2c9 adding support for B2 region (#4177)
Co-authored-by: Muhammad Hallaj bin Subery <hallaj@tuta.io>
2023-02-05 21:24:21 -08:00
CommanderRoot
c7c9d22f37 filer.store.mysql: Use utf8mb4 instead of 3 byte UTF8 (#4094) 2023-01-01 05:07:53 -08:00
CommanderRoot
c2280e94cf filer.store.mysql: Replace deprecated upsert syntax (#4096) 2023-01-01 05:06:57 -08:00
Ryan Russell
f6d391c9da docs(command): readability fixes (#3686)
Signed-off-by: Ryan Russell <git@ryanrussell.org>

Signed-off-by: Ryan Russell <git@ryanrussell.org>
2022-09-15 01:33:04 -07:00
Konstantin Lebedev
7e09a548a6 exclude directories to sync on filer 2022-07-27 19:22:57 +05:00
duanhongyi
1ceab96aba filer tikv support tls 2022-07-08 14:23:06 +08:00
chrislu
b7de5c6c43 shell: remove unused filer parameter from shell.toml 2022-07-07 15:01:23 -07:00
yulai.li
46e0b629e5 Update tikv client version and add one PC support 2022-06-26 22:43:37 +08:00
chrislu
29198720f2 s3: add grpc server to accept configuration changes 2022-05-15 00:43:37 -07:00
Konstantin Lebedev
f127b326bf add options to scaffold 2022-05-03 22:54:31 +05:00
Konstantin Lebedev
d8925b4e83 Merge branch 'new_master' into ydb
# Conflicts:
#	go.mod
#	go.sum
2022-05-03 00:13:57 +05:00
Konstantin Lebedev
50c4f62ed4 ydb-go-sdk move to v3 2022-05-02 02:07:47 +05:00
chrislu
192983b464 s3 backend support customizing storage class 2022-04-30 17:36:40 -07:00
a
7e92517571 change user and pass to username and password 2022-04-01 14:09:25 -05:00
elee
423ce57cde prefix search, bucket implemented 2022-03-17 21:12:25 -05:00
elee
921535001a arangodb adapter 2022-03-17 04:49:26 -05:00
Berck Nash
9b14f0c81a Add mTLS support for both master and volume http server. 2022-03-16 09:52:17 -06:00
banjiaojuhao
b5ec346700 FilerStore: add redis_lua 2022-02-15 20:54:57 +08:00
guol-fnst
da9540e666 add gocql timeout setting 2022-01-18 15:21:13 +08:00
chrislu
826a7b307e master: remove hard coded filer settings in master.toml
fix https://github.com/chrislusf/seaweedfs/issues/2529
2022-01-12 01:11:25 -08:00
Sebastian Kurfuerst
1cd3b6b4e1 BUGFIX: security.toml contained wrong keys 2021-12-31 22:05:41 +01:00