Lisandro Pin
|
6b98b52acc
|
Fix reporting of EC shard sizes from nodes to masters. (#7835)
SeaweedFS tracks EC shard sizes on topology data stuctures, but this information is never
relayed to master servers :( The end result is that commands reporting disk usage, such
as `volume.list` and `cluster.status`, yield incorrect figures when EC shards are present.
As an example for a simple 5-node test cluster, before...
```
> volume.list
Topology volumeSizeLimit:30000 MB hdd(volume:6/40 active:6 free:33 remote:0)
DataCenter DefaultDataCenter hdd(volume:6/40 active:6 free:33 remote:0)
Rack DefaultRack hdd(volume:6/40 active:6 free:33 remote:0)
DataNode 192.168.10.111:9001 hdd(volume:1/8 active:1 free:7 remote:0)
Disk hdd(volume:1/8 active:1 free:7 remote:0) id:0
volume id:3 size:88967096 file_count:172 replica_placement:2 version:3 modified_at_second:1766349617
ec volume id:1 collection: shards:[1 5]
Disk hdd total size:88967096 file_count:172
DataNode 192.168.10.111:9001 total size:88967096 file_count:172
DataCenter DefaultDataCenter hdd(volume:6/40 active:6 free:33 remote:0)
Rack DefaultRack hdd(volume:6/40 active:6 free:33 remote:0)
DataNode 192.168.10.111:9002 hdd(volume:2/8 active:2 free:6 remote:0)
Disk hdd(volume:2/8 active:2 free:6 remote:0) id:0
volume id:2 size:77267536 file_count:166 replica_placement:2 version:3 modified_at_second:1766349617
volume id:3 size:88967096 file_count:172 replica_placement:2 version:3 modified_at_second:1766349617
ec volume id:1 collection: shards:[0 4]
Disk hdd total size:166234632 file_count:338
DataNode 192.168.10.111:9002 total size:166234632 file_count:338
DataCenter DefaultDataCenter hdd(volume:6/40 active:6 free:33 remote:0)
Rack DefaultRack hdd(volume:6/40 active:6 free:33 remote:0)
DataNode 192.168.10.111:9003 hdd(volume:1/8 active:1 free:7 remote:0)
Disk hdd(volume:1/8 active:1 free:7 remote:0) id:0
volume id:2 size:77267536 file_count:166 replica_placement:2 version:3 modified_at_second:1766349617
ec volume id:1 collection: shards:[2 6]
Disk hdd total size:77267536 file_count:166
DataNode 192.168.10.111:9003 total size:77267536 file_count:166
DataCenter DefaultDataCenter hdd(volume:6/40 active:6 free:33 remote:0)
Rack DefaultRack hdd(volume:6/40 active:6 free:33 remote:0)
DataNode 192.168.10.111:9004 hdd(volume:2/8 active:2 free:6 remote:0)
Disk hdd(volume:2/8 active:2 free:6 remote:0) id:0
volume id:2 size:77267536 file_count:166 replica_placement:2 version:3 modified_at_second:1766349617
volume id:3 size:88967096 file_count:172 replica_placement:2 version:3 modified_at_second:1766349617
ec volume id:1 collection: shards:[3 7]
Disk hdd total size:166234632 file_count:338
DataNode 192.168.10.111:9004 total size:166234632 file_count:338
DataCenter DefaultDataCenter hdd(volume:6/40 active:6 free:33 remote:0)
Rack DefaultRack hdd(volume:6/40 active:6 free:33 remote:0)
DataNode 192.168.10.111:9005 hdd(volume:0/8 active:0 free:8 remote:0)
Disk hdd(volume:0/8 active:0 free:8 remote:0) id:0
ec volume id:1 collection: shards:[8 9 10 11 12 13]
Disk hdd total size:0 file_count:0
Rack DefaultRack total size:498703896 file_count:1014
DataCenter DefaultDataCenter total size:498703896 file_count:1014
total size:498703896 file_count:1014
```
...and after:
```
> volume.list
Topology volumeSizeLimit:30000 MB hdd(volume:6/40 active:6 free:33 remote:0)
DataCenter DefaultDataCenter hdd(volume:6/40 active:6 free:33 remote:0)
Rack DefaultRack hdd(volume:6/40 active:6 free:33 remote:0)
DataNode 192.168.10.111:9001 hdd(volume:1/8 active:1 free:7 remote:0)
Disk hdd(volume:1/8 active:1 free:7 remote:0) id:0
volume id:2 size:81761800 file_count:161 replica_placement:2 version:3 modified_at_second:1766349495
ec volume id:1 collection: shards:[1 5 9] sizes:[1:8.00 MiB 5:8.00 MiB 9:8.00 MiB] total:24.00 MiB
Disk hdd total size:81761800 file_count:161
DataNode 192.168.10.111:9001 total size:81761800 file_count:161
DataCenter DefaultDataCenter hdd(volume:6/40 active:6 free:33 remote:0)
Rack DefaultRack hdd(volume:6/40 active:6 free:33 remote:0)
DataNode 192.168.10.111:9002 hdd(volume:1/8 active:1 free:7 remote:0)
Disk hdd(volume:1/8 active:1 free:7 remote:0) id:0
volume id:3 size:88678712 file_count:170 replica_placement:2 version:3 modified_at_second:1766349495
ec volume id:1 collection: shards:[11 12 13] sizes:[11:8.00 MiB 12:8.00 MiB 13:8.00 MiB] total:24.00 MiB
Disk hdd total size:88678712 file_count:170
DataNode 192.168.10.111:9002 total size:88678712 file_count:170
DataCenter DefaultDataCenter hdd(volume:6/40 active:6 free:33 remote:0)
Rack DefaultRack hdd(volume:6/40 active:6 free:33 remote:0)
DataNode 192.168.10.111:9003 hdd(volume:2/8 active:2 free:6 remote:0)
Disk hdd(volume:2/8 active:2 free:6 remote:0) id:0
volume id:2 size:81761800 file_count:161 replica_placement:2 version:3 modified_at_second:1766349495
volume id:3 size:88678712 file_count:170 replica_placement:2 version:3 modified_at_second:1766349495
ec volume id:1 collection: shards:[0 4 8] sizes:[0:8.00 MiB 4:8.00 MiB 8:8.00 MiB] total:24.00 MiB
Disk hdd total size:170440512 file_count:331
DataNode 192.168.10.111:9003 total size:170440512 file_count:331
DataCenter DefaultDataCenter hdd(volume:6/40 active:6 free:33 remote:0)
Rack DefaultRack hdd(volume:6/40 active:6 free:33 remote:0)
DataNode 192.168.10.111:9004 hdd(volume:2/8 active:2 free:6 remote:0)
Disk hdd(volume:2/8 active:2 free:6 remote:0) id:0
volume id:2 size:81761800 file_count:161 replica_placement:2 version:3 modified_at_second:1766349495
volume id:3 size:88678712 file_count:170 replica_placement:2 version:3 modified_at_second:1766349495
ec volume id:1 collection: shards:[2 6 10] sizes:[2:8.00 MiB 6:8.00 MiB 10:8.00 MiB] total:24.00 MiB
Disk hdd total size:170440512 file_count:331
DataNode 192.168.10.111:9004 total size:170440512 file_count:331
DataCenter DefaultDataCenter hdd(volume:6/40 active:6 free:33 remote:0)
Rack DefaultRack hdd(volume:6/40 active:6 free:33 remote:0)
DataNode 192.168.10.111:9005 hdd(volume:0/8 active:0 free:8 remote:0)
Disk hdd(volume:0/8 active:0 free:8 remote:0) id:0
ec volume id:1 collection: shards:[3 7] sizes:[3:8.00 MiB 7:8.00 MiB] total:16.00 MiB
Disk hdd total size:0 file_count:0
Rack DefaultRack total size:511321536 file_count:993
DataCenter DefaultDataCenter total size:511321536 file_count:993
total size:511321536 file_count:993
```
|
2025-12-28 19:30:42 -08:00 |
|
Lisandro Pin
|
dddb0f0ae5
|
Fix update of SeaweedFS_volumeServer_volumes gauge metrics when EC shards are unmounted (#6776)
|
2025-05-09 10:15:34 -07:00 |
|
chrislu
|
c9f3448692
|
ReadAt may return io.EOF t end of file
related to https://github.com/seaweedfs/seaweedfs/issues/6219
|
2024-11-21 00:37:38 -08:00 |
|
steve.wei
|
0bdf121e51
|
rename VolumeServerVolumeGauge (#5504)
|
2024-04-17 04:49:50 -07:00 |
|
chrislu
|
6ebe26a765
|
Revert "Revert "Revert "Add disk type to prometheus metrics" (#4777)""
This reverts commit 567d788928.
|
2023-10-03 08:28:52 -07:00 |
|
chrislu
|
7540d43ee9
|
Revert "Revert "fix compilation""
This reverts commit f9abfd0b03.
|
2023-10-03 08:28:02 -07:00 |
|
chrislu
|
249c0e06ef
|
Revert "fix compilation"
This reverts commit 451ec6504d.
|
2023-10-03 08:27:50 -07:00 |
|
chrislu
|
451ec6504d
|
fix compilation
|
2023-10-03 08:15:18 -07:00 |
|
chrislu
|
f9abfd0b03
|
Revert "fix compilation"
This reverts commit 0483ba3889.
|
2023-10-03 07:58:11 -07:00 |
|
chrislu
|
0483ba3889
|
fix compilation
|
2023-10-02 11:55:50 -07:00 |
|
chrislu
|
567d788928
|
Revert "Revert "Add disk type to prometheus metrics" (#4777)"
This reverts commit 9215ba24be.
|
2023-10-02 11:49:54 -07:00 |
|
Nikita Mochalov
|
e6a49dc533
|
Fix resource leaks (#4737)
* Fix division by zero
* Fix file handle leak
* Fix file handle leak
* Fix file handle leak
* Fix goroutine leak
|
2023-08-09 15:30:36 -07:00 |
|
chrislu
|
26dbc6c905
|
move to https://github.com/seaweedfs/seaweedfs
|
2022-07-29 00:17:28 -07:00 |
|
Chris Lu
|
f8446b42ab
|
this can compile now!!!
|
2021-02-16 02:47:02 -08:00 |
|
Chris Lu
|
ab759f0ec2
|
erasure coding: fix EC error if multiple disks are configured in one volume server
|
2020-08-26 09:16:58 -07:00 |
|
Chris Lu
|
d1cf39f180
|
fix logging
|
2020-08-26 08:55:15 -07:00 |
|
Chris Lu
|
a7b1b23c58
|
fix wrong volume count
fix https://github.com/chrislusf/seaweedfs/issues/1013
|
2019-07-17 00:03:15 -07:00 |
|
Chris Lu
|
115558e5f5
|
adjust counters
|
2019-06-17 21:02:50 -07:00 |
|
Chris Lu
|
289fd7eb39
|
count number of volumes and ec shards
|
2019-06-16 02:24:15 -07:00 |
|
Chris Lu
|
ca8a2bb534
|
go fmt
|
2019-06-04 22:04:10 -07:00 |
|
Chris Lu
|
2215e81be7
|
ui add ec shard statuses
|
2019-06-04 21:52:37 -07:00 |
|
Chris Lu
|
7e80b2b882
|
fix multiple bugs
|
2019-06-03 02:26:31 -07:00 |
|
Chris Lu
|
40ca2f2903
|
add collection.delete
|
2019-05-30 09:47:54 -07:00 |
|
Chris Lu
|
3a8c1055a2
|
refactoring ecx to ecVolume
|
2019-05-27 22:00:36 -07:00 |
|
Chris Lu
|
b4b407e403
|
add grpc ec shard read
|
2019-05-27 11:59:03 -07:00 |
|
Chris Lu
|
a4f3d82c57
|
convert needle id to ec intervals to read from
|
2019-05-27 01:29:46 -07:00 |
|