Commit Graph

21 Commits

Author SHA1 Message Date
promalert
9012069bd7 chore: execute goimports to format the code (#7983)
* chore: execute goimports to format the code

Signed-off-by: promalert <promalert@outlook.com>

* goimports -w .

---------

Signed-off-by: promalert <promalert@outlook.com>
Co-authored-by: Chris Lu <chris.lu@gmail.com>
2026-01-07 13:06:08 -08:00
Chris Lu
84b8a8e010 filer.sync: fix checkpoint not being saved properly (#7719)
* filer.sync: fix race condition on first checkpoint save

Initialize lastWriteTime to time.Now() instead of zero time to prevent
the first checkpoint save from being triggered immediately when the
first event arrives. This gives async jobs time to complete and update
the watermark before the checkpoint is saved.

Previously, the zero time caused lastWriteTime.Add(3s).Before(now) to
be true on the first event, triggering an immediate checkpoint save
attempt. But since jobs are processed asynchronously, the watermark
was still 0 (initial value), causing the save to be skipped due to
the 'if offsetTsNs == 0 { return nil }' check.

Fixes #7717

* filer.sync: save checkpoint on graceful shutdown

Add graceful shutdown handling to save the final checkpoint when
filer.sync is terminated. Previously, any sync progress within the
last 3-second checkpoint interval would be lost on shutdown.

Changes:
- Add syncState struct to track current processor and offset save info
- Add atomic pointers syncStateA2B and syncStateB2A for both directions
- Register grace.OnInterrupt hook to save checkpoints on shutdown
- Modify doSubscribeFilerMetaChanges to update sync state atomically

This ensures that when filer.sync is restarted, it resumes from the
correct position instead of potentially replaying old events.

Fixes #7717
2025-12-11 10:25:02 -08:00
Chris Lu
69553e5ba6 convert error fromating to %w everywhere (#6995) 2025-07-16 23:39:27 -07:00
Konstantin Lebedev
7988ee0805 [fs.verify] skip failed files if entry not found on filerStore (#5693) 2024-06-19 06:50:55 -07:00
chrislu
b02fdeabff RetryForever => RetryUntil 2023-10-01 11:33:56 -07:00
chrislu
5db9fcccd4 refactoring 2023-03-21 23:01:49 -07:00
chrislu
81fdf3651b grpc connection to filer add sw-client-id header 2023-01-20 01:48:12 -08:00
chrislu
31922b2bf2 s3 to watch specific directories 2022-09-20 09:30:05 -07:00
chrislu
26dbc6c905 move to https://github.com/seaweedfs/seaweedfs 2022-07-29 00:17:28 -07:00
chrislu
64f3d6fb6e metadata subscription uses client epoch 2022-07-23 10:50:28 -07:00
creeew
02ae102731 fix filer.sync missing source srv uploaded files to target when target down 2022-06-02 01:28:47 +08:00
chrislu
a2b101a737 subscribe metadata between a range 2022-05-30 15:04:19 -07:00
chrislu
8bc16f1149 rename 2022-05-06 23:38:22 -07:00
chrislu
6a12520a96 fix logging 2022-01-10 01:00:11 -08:00
chrislu
5c87fcc6d2 add client id for all metadata listening clients 2021-12-30 00:23:57 -08:00
chrislu
9f9ef1340c use streaming mode for long poll grpc calls
streaming mode would create separate grpc connections for each call.
this is to ensure the long poll connections are properly closed.
2021-12-26 00:15:03 -08:00
Chris Lu
004e56c1a6 save updated lastTsNs 2021-10-17 02:01:20 -07:00
Chris Lu
e5fc35ed0c change server address from string to a type 2021-09-12 22:47:52 -07:00
Chris Lu
8e125339d5 line wrap 2021-08-31 23:30:28 -07:00
Chris Lu
68ea99190f add additionalPathPrefixes 2021-08-31 23:26:26 -07:00
Chris Lu
6b743dbbf9 refactor client subscribe metadata 2021-08-04 16:25:46 -07:00