gsh-digital-services/seaweedFS
5
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
13,366 Commits 1 Branch 0 Tags
master
Commit Graph

28 Commits

Author SHA1 Message Date
Chris Lu
be6b5db65a s3: fix health check endpoints returning 404 for HEAD requests #8243 (#8248) 
* Fix disk errors handling in vacuum compaction

When a disk reports IO errors during vacuum compaction (e.g., 'read /mnt/d1/weed/oc_xyz.dat: input/output error'), the vacuum task should signal the error to the master so it can:
1. Drop the faulty volume replica
2. Rebuild the replica from healthy copies

Changes:
- Add checkReadWriteError() calls in vacuum read paths (ReadNeedleBlob, ReadData, ScanVolumeFile) to flag EIO errors in volume.lastIoError
- Preserve error wrapping using %w format instead of %v so EIO propagates correctly
- The existing heartbeat logic will detect lastIoError and remove the bad volume

Fixes issue #8237

* error

* s3: fix health check endpoints returning 404 for HEAD requests #8243
 2026-02-08 19:08:10 -08:00
Chris Lu
330ba7d9dc Fix disk errors handling in vacuum compaction (#8244) 
When a disk reports IO errors during vacuum compaction (e.g., 'read /mnt/d1/weed/oc_xyz.dat: input/output error'), the vacuum task should signal the error to the master so it can:
1. Drop the faulty volume replica
2. Rebuild the replica from healthy copies

Changes:
- Add checkReadWriteError() calls in vacuum read paths (ReadNeedleBlob, ReadData, ScanVolumeFile) to flag EIO errors in volume.lastIoError
- Preserve error wrapping using %w format instead of %v so EIO propagates correctly
- The existing heartbeat logic will detect lastIoError and remove the bad volume

Fixes issue #8237
 2026-02-07 21:33:02 -08:00
Chris Lu
bf8e4f40e6 S3: Perf related (#7463) 
* reduce checks

* s3 object lookup optimization

* Only check versioning configuration if client requests
* Consolidate SSE Entry Lookups

* optimize

* revert optimization for versioned objects

* Removed: getObjectEntryForSSE() function

* refactor

* Refactoring: Added fetchObjectEntryRequired

* avoid refetching

* return early if not found

* reuse objects from conditional check

* clear cache when creating bucket
 2025-11-10 20:30:21 -08:00
Chris Lu
69553e5ba6 convert error fromating to %w everywhere (#6995) 2025-07-16 23:39:27 -07:00
chrislu
2f1b3d68d7 pass volume version when creating a volume 2025-06-19 01:15:25 -07:00
Eugeniy E. Mikhailov
3003c9e17e added backward compatible CRC check and extra explanatory comments (#6233) 2024-11-12 13:56:14 -08:00
Eugeniy E. Mikhailov
bc01f09e37 Do CRC check if the buffer contains the full needle data before it is sent (#5980) 2024-09-05 17:14:55 -07:00
Eugeniy E. Mikhailov
220061573d update metric for ErrorCRC if it happens during volume readNeedleDataInto (#5957) 2024-09-01 20:30:09 -07:00
skycope
316ae45795 can vacuum volume when size mismatch (#5200) 
Co-authored-by: Yang Wang <yangwang@weride.ai>
 2024-01-15 16:16:46 -08:00
Eric Yang
51d462f204 ADHOC: volume fsck using append at ns (#3906) 
* ADHOC: volume fsck using append at ns

* nit

* nit

Co-authored-by: root <root@HQ-10MSTD3EY.roblox.local>
 2022-10-24 22:09:38 -07:00
Eric Yang
56c94cc08e ADHOC: filter deleted files from idx file binary search (#3763) 
* ADHOC: filter deleted files from idx file binary search

* remove unwanted check

Co-authored-by: root <root@HQ-10MSTD3EY.roblox.local>
 2022-09-29 12:48:36 -07:00
chrislu
3fc261d27c Merge branch 'master' of https://github.com/seaweedfs/seaweedfs 2022-09-16 23:54:39 -07:00
chrislu
06748f2771 remove unused code 2022-09-16 23:54:37 -07:00
famosss
d949a238b8 volume: add "readBufSize" option to customize read optimization (#3702) 
* simplify a bit

* feat: volume: add "readBufSize" option to customize read optimization

* refactor : redbufSIze -> readBufferSize

* simplify a bit

* simplify a bit
 2022-09-16 00:30:40 -07:00
chrislu
896a85d6e4 volume: add "hasSlowRead" option to customize read optimization 2022-09-15 03:11:32 -07:00
famosss
41af5a6ee5 simplify a bit (#3631) 2022-09-08 21:01:21 -07:00
chrislu
64e75a286e simplify a bit 2022-09-08 19:07:57 -07:00
chrislu
d14d029c73 reduce v.dataFileAccessLock lock scope 2022-09-08 19:05:31 -07:00
chrislu
406a80da4b adjust variable names 2022-09-08 18:54:16 -07:00
chrislu
2bfc8970d2 refactor: move ReadNeedleDataInto into volume_read.go 2022-09-08 18:54:02 -07:00
Eric Yang
b324a6536c ADHOC: add read needle meta grpc (#3581) 
* ADHOC: add read needle meta grpc

* add test

* nit

Co-authored-by: root <root@HQ-10MSTD3EY.roblox.local>
 2022-09-06 23:51:27 -07:00
chrislu
8c5759a06d minor optimization 2022-09-04 18:50:45 -07:00
chrislu
26dbc6c905 move to https://github.com/seaweedfs/seaweedfs 2022-07-29 00:17:28 -07:00
chrislu
056c480eb0 volume: checksum remove the hashing step in Value() 2022-06-05 15:24:02 -07:00
chrislu
ecef844dfc stream read large files 2022-06-05 11:54:04 -07:00
Chris Lu
734c980040 volume: support concurrent download data size limit 2021-08-08 23:25:16 -07:00
Chris Lu
b465095db1 shell: add volume.check.disk to fix inconsistency for replicated volumes 
fix https://github.com/chrislusf/seaweedfs/issues/1923
 2021-03-22 00:03:16 -07:00
Chris Lu
102a951377 refactor, split into 2 files 2021-03-21 13:05:53 -07:00