gsh-digital-services/seaweedFS
5
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
11,095 Commits 1 Branch 0 Tags
e2d30cc3bc05b225fed1a111b4a403b9dbff5aba
Commit Graph

7151 Commits

Author SHA1 Message Date
wyang
a7973ed7d1 fix deadlock hang when broadcast to clients (#6184) 
fix deadlock when broadcast to clients

when master thransfer leader, the old master will disconnect with all
filers and volumeServers, if the cluster is a big , the broadcast
messages may be more big than the max of the channel len 100, then if the
KeepConnect was not listen on the channel in disconnect, it will
deadlock. and the whole cluster will not serve!
 2024-11-03 23:20:48 -08:00
Konstantin Lebedev
5bddf0c085 [shell] volume.balance collect volume servers by dc rack node (#6191) 
* chore: balance by rack

* fix: rm check lock

* fix: selected racks

* fix: selected nodes

* fix: containts

* fix: one collectVolumeServersByDcRackNode

* fix: revert lock and add lock

* fix: panic test

* revert noLock
 2024-11-03 11:08:45 -08:00
han
65fb8fad99 Update filer_multipart.go (#6188) 
fix the #6177 bug
 2024-11-01 00:38:35 -07:00
Bruce
0060a2cf9c Fix 6181/6182 (#6183) 
* set larger buf size for LogBuffer

* jump to next day when no more entry found

* Update weed/filer/filer_notify_read.go

---------

Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
 2024-10-31 08:40:05 -07:00
wyang
c29c912bdc fix format (#6185) 
unitest weed/shell fail
 2024-10-31 08:30:35 -07:00
chrislu
ad6240f066 fix parameter 2024-10-30 20:30:51 -07:00
chrislu
8802843511 pass in bytes buffer to avoid possible race condition 
related to https://github.com/seaweedfs/seaweedfs/issues/6180
 2024-10-30 09:31:22 -07:00
chrislu
3feb66d0a1 adding more debug message 
related to https://github.com/seaweedfs/seaweedfs/issues/6180
 2024-10-30 09:16:40 -07:00
chrislu
228946369c 3.79 2024-10-28 19:44:39 -07:00
chrislu
9873b033d1 backward compatible vif loading 2024-10-28 19:44:30 -07:00
chrislu
9105c6bdd1 fix format 2024-10-28 11:29:08 -07:00
chrislu
ebbb35d533 3.78 2024-10-27 17:44:22 -07:00
chrislu
089d4316ef ensure 2 volume space since actual need 1.4x volume size empty space 2024-10-24 22:44:53 -07:00
chrislu
6e388e29c9 correcting free volume count, factor it during ec encoding to ensure enough disk space available 
fix https://github.com/seaweedfs/seaweedfs/issues/6163
 2024-10-24 22:42:38 -07:00
chrislu
8d6189bcc5 adjust output format 2024-10-24 21:41:39 -07:00
chrislu
2f3d820f52 rename proto field 
This should not have any impact.
 2024-10-24 21:36:56 -07:00
chrislu
ae5bd0667a rename proto field from DestroyTime to expire_at_sec 
For TTL volume converted into EC volume, this change may leave the volumes staying.
 2024-10-24 21:35:11 -07:00
chrislu
1c2b10303a remove X-Forwarded-For 
fix https://github.com/seaweedfs/seaweedfs/issues/6136
 2024-10-23 17:52:45 -07:00
chrislu
73921ce4f6 adjust help message 2024-10-22 08:51:02 -07:00
Guang Jiong Lou
9369a88c5c stop renaming worm files (#6154) 
* stop renaming worm file

Signed-off-by: lou <alex1988@outlook.com>

* update after review

Signed-off-by: lou <alex1988@outlook.com>

* Update weed/server/filer_server_handlers_write.go

---------

Signed-off-by: lou <alex1988@outlook.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
 2024-10-22 08:46:18 -07:00
chrislu
d38456dc06 do not delete collection metrics when unmount a volume 
fix https://github.com/seaweedfs/seaweedfs/issues/6139
 2024-10-22 00:44:52 -07:00
chrislu
6c7824c813 consistent sorted tags elastic,gocdk,rclone,sqlite,tikv,ydb 2024-10-21 22:53:29 -07:00
chrislu
0df4a4298b adjust comment 2024-10-21 22:36:25 -07:00
Guang Jiong Lou
54b91ba2b2 fix permission check in put request (#6142) 2024-10-18 07:00:45 -07:00
dsd
1e13b6879c fix(volume): to avoid duplicate write a same needle (#6138) 
fix WriteNeedleBlob to avoid duplicate write a same needle

Co-authored-by: 邓书东 <shudong_deng@hhnb2024010108.intsig.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
 2024-10-18 01:20:50 -07:00
chrislu
f11567816e Revert "fix: volume data integrity check (#6118)" 
This reverts commit e12b9ab746.
 2024-10-17 18:52:30 -07:00
wusong
e12b9ab746 fix: volume data integrity check (#6118) 
Co-authored-by: wang wusong <wangwusong@virtaitech.com>
 2024-10-14 12:17:41 -07:00
steve.wei
cfbe45c765 feat: add in-flight metric for s3/file/volume-server (#6120) 2024-10-14 12:10:05 -07:00
chrislu
804e765397 remove unused 2024-10-13 23:12:10 -07:00
chrislu
6845e25318 set proxied request content length 2024-10-10 13:55:32 -07:00
chrislu
35fd1e1c9a optimize memory usage for large number of volumes 
1. unwrap the map to avoid extra map object creation
2. fix ec shard counting in UpdateEcShards
 2024-10-10 10:00:30 -07:00
chrislu
b28b1a3402 3.77 2024-10-10 01:28:06 -07:00
chrislu
07df5c53de avoid nil 
I1010 03:10:25.220345 mount_std.go:281 This is SeaweedFS version 30GB 3.76 82ed61c655 linux amd64
^[OR^[OR2024/10/10 03:12:56 Unimplemented opcode POLL
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x28 pc=0x3983b41]
goroutine 112 [running]:
github.com/seaweedfs/seaweedfs/weed/mount/meta_cache.SubscribeMetaEvents.func2.mergeProceesors.1(0xc000513540)
/github/workspace/weed/mount/meta_cache/meta_cache_subscribe.go:30 +0x41
github.com/seaweedfs/seaweedfs/weed/pb.WithFilerClientFollowMetadata.makeSubscribeMetadataFunc.func1({0x514a560, 0xc0009b06d0})
/github/workspace/weed/pb/filer_pb_tail.go:87 +0x293
github.com/seaweedfs/seaweedfs/weed/mount.(*WFS).WithFilerClient.func1.1(0xc000d9e008)
/github/workspace/weed/mount/wfs_filer_client.go:27 +0x68
github.com/seaweedfs/seaweedfs/weed/pb.WithGrpcClient(0xa9?, 0x9d1f90fb?, 0xc00098fbe0, {0xc000e1c078, 0x15}, 0x0?, {0xc000a76bd0?, 0x1?, 0x1?
})
/github/workspace/weed/pb/grpc_client_server.go:155 +0x33a
github.com/seaweedfs/seaweedfs/weed/mount.(*WFS).WithFilerClient.func1()
/github/workspace/weed/mount/wfs_filer_client.go:25 +0x14c
github.com/seaweedfs/seaweedfs/weed/util.Retry({0x489a10c, 0xa}, 0xc00098fcf8)
/github/workspace/weed/util/retry.go:16 +0xa7
github.com/seaweedfs/seaweedfs/weed/mount.(*WFS).WithFilerClient(0x0?, 0x30?, 0xc00056a568?)
/github/workspace/weed/mount/wfs_filer_client.go:18 +0x65
github.com/seaweedfs/seaweedfs/weed/pb.WithFilerClientFollowMetadata({0x50f5898, 0xc000e2c180}, 0xc00101e000, 0xc000d22030)
/github/workspace/weed/pb/filer_pb_tail.go:51 +0x87
github.com/seaweedfs/seaweedfs/weed/mount/meta_cache.SubscribeMetaEvents.func2()
/github/workspace/weed/mount/meta_cache/meta_cache_subscribe.go:115 +0xcb
github.com/seaweedfs/seaweedfs/weed/util.RetryUntil({0x48b8b35, 0x11}, 0xc00098fef0, 0x4af0048)
/github/workspace/weed/util/retry.go:64 +0x4e
github.com/seaweedfs/seaweedfs/weed/mount/meta_cache.SubscribeMetaEvents(0xc000d1d980, 0x9d1f90fb, {0x50f5898, 0xc000e2c180}, {0x7ffdf6f98dca,
0x4}, 0x17fcf8b7c7ecf8cf, {0xc0000ac208, 0x1, 0x1})
/github/workspace/weed/mount/meta_cache/meta_cache_subscribe.go:113 +0x39b
created by github.com/seaweedfs/seaweedfs/weed/mount.(*WFS).StartBackgroundTasks in goroutine 1
/github/workspace/weed/mount/weedfs.go:150 +0x146
 2024-10-09 22:46:00 -07:00
chrislu
82ed61c655 3.76 2024-10-06 19:37:03 -07:00
chrislu
c0e36231ad use only one metadata follow process 2024-10-06 12:55:19 -07:00
Konstantin Lebedev
44b275879b [s3] add {Get,Put,Delete}BucketTagging and PublicAccessBlock Handlers (#6088) 
* add {Get,Put,Delete}BucketTagging Handlers

* s3 add skip bucket PublicAccessBlock handlers

---------

Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
 2024-10-04 10:59:14 -07:00
chrislu
a3a8f6217e fix 2024-10-03 09:03:17 -07:00
chrislu
117c3aebda 3.75 2024-10-03 08:33:35 -07:00
Konstantin Lebedev
a3556acd98 [s3] add skip bucket encryption handlers (#6091) 
s3 add skip bucket encryption handlers
 2024-10-03 08:10:43 -07:00
Er2
5644bc8f01 s3api: Fix signature v4 with reverse proxy at sub-path (#6092) 2024-10-03 08:08:19 -07:00
chrislu
ff47956530 add helper functions 2024-10-03 00:13:46 -07:00
chrislu
11e53dd8ef skip error if conf file is not created 2024-10-01 22:46:48 -07:00
chrislu
f747767aa1 support load balancer in front of s3 2024-10-01 12:57:45 -07:00
chrislu
7dd480acad handle missing leading zero padding in replica 2024-10-01 10:16:21 -07:00
chrislu
e3fa87bcc1 3.74 2024-09-30 11:29:01 -07:00
chrislu
cdd7fa81ab fix help message 2024-09-29 21:53:07 -07:00
chrislu
20929f2a57 adjust resource heavy for volume.fix.replication 2024-09-29 11:32:18 -07:00
chrislu
6564ceda91 skip resource heavy commands from running on master nodes 2024-09-29 10:51:17 -07:00
chrislu
ec30a504ba refactor 2024-09-29 10:38:22 -07:00
chrislu
9cd263b2ce refactor 2024-09-29 10:35:53 -07:00