Commit Graph

26 Commits

Author SHA1 Message Date
Chris Lu
6442da6f17 mount: efficient file lookup in large directories, skipping directory caching (#7818)
* mount: skip directory caching on file lookup and write

When opening or creating a file in a directory that hasn't been cached yet,
don't list the entire directory. Instead:
- For reads: fetch only the single file's metadata directly from the filer
- For writes: create on filer but skip local cache insertion

This fixes a performance issue where opening a file in a directory
with millions of files would hang because EnsureVisited() had to
list all entries before the open could complete.

The directory will still be cached when explicitly listed (ReadDir),
but individual file operations now bypass the full directory caching.

Key optimizations:
- Extract shared lookupEntry() method to eliminate code duplication
- Skip EnsureVisited on Lookup (file open)
- Skip cache insertion on Mknod, Mkdir, Symlink, Link if dir not cached
- Skip cache update on file sync/flush if dir not cached
- If directory IS cached and entry not found, return ENOENT immediately

Fixes #7145

* mount: add error handling for meta cache insert/update operations

Handle errors from metaCache.InsertEntry and metaCache.UpdateEntry calls
instead of silently ignoring them. This prevents silent cache inconsistencies
and ensures errors are properly propagated.

Files updated:
- filehandle_read.go: handle InsertEntry error in downloadRemoteEntry
- weedfs_file_sync.go: handle InsertEntry error in doFlush
- weedfs_link.go: handle UpdateEntry and InsertEntry errors in Link
- weedfs_symlink.go: handle InsertEntry error in Symlink

* mount: use error wrapping (%w) for consistent error handling

Use %w instead of %v in fmt.Errorf to preserve the original error,
allowing it to be inspected up the call stack with errors.Is/As.
2025-12-18 21:19:15 -08:00
Chris Lu
ec3378f7a6 fix: improve mount quota enforcement to prevent overflow (#7804)
* fix: improve mount quota enforcement to prevent overflow (fixes seaweedfs-csi-driver#218)

* test: add unit tests for quota enforcement
2025-12-17 01:14:01 -08:00
Aleksey Kosov
165af32d6b added context to filer_client method calls (#6808)
Co-authored-by: akosov <a.kosov@kryptonite.ru>
2025-05-22 09:46:49 -07:00
Guang Jiong Lou
3b1ac77e1f worm grace period and retention time support (#6404)
Signed-off-by: lou <alex1988@outlook.com>
2024-12-31 18:41:43 -08:00
Guang Jiong Lou
6c986e9d70 improve worm support (#5983)
* improve worm support

Signed-off-by: lou <alex1988@outlook.com>

* worm mode in filer

Signed-off-by: lou <alex1988@outlook.com>

* update after review

Signed-off-by: lou <alex1988@outlook.com>

* update after review

Signed-off-by: lou <alex1988@outlook.com>

* move to fs configure

Signed-off-by: lou <alex1988@outlook.com>

* remove flag

Signed-off-by: lou <alex1988@outlook.com>

* update after review

Signed-off-by: lou <alex1988@outlook.com>

* support worm hardlink

Signed-off-by: lou <alex1988@outlook.com>

* update after review

Signed-off-by: lou <alex1988@outlook.com>

* typo

Signed-off-by: lou <alex1988@outlook.com>

* sync filer conf

Signed-off-by: lou <alex1988@outlook.com>

---------

Signed-off-by: lou <alex1988@outlook.com>
2024-09-16 21:02:21 -07:00
wusong
665917fdf1 [mount]: delete redundant code (#4173) 2023-01-31 18:54:46 -08:00
chrislu
26dbc6c905 move to https://github.com/seaweedfs/seaweedfs 2022-07-29 00:17:28 -07:00
chrislu
4fd5f96598 filer: remove replication, collection, disk_type info from entry metadata
these metadata can change and are not used
2022-06-06 00:39:35 -07:00
chrislu
7542fd7f4d mount: optimize a bit when writing files 2022-03-17 00:02:38 -07:00
chrislu
bd5c5586b5 generate inode via path and time 2022-03-14 00:03:29 -07:00
chrislu
f3442e36e6 mount: quota adjust error type to be syscall.ENOSPC 2022-03-06 17:04:21 -08:00
chrislu
b7c992f410 add flag to enforce quota 2022-03-05 22:10:43 -08:00
chrislu
63a9d8f01d ensure inodes are not duplicating unless hardlinked 2022-02-27 23:13:49 -08:00
chrislu
be3fc77391 mount2: use consistent inode 2022-02-25 00:53:27 -08:00
chrislu
ceaf993a27 mount2: add rdev 2022-02-24 14:51:25 -08:00
chrislu
91f0481f4e mount2: SetAttr set mode correctly 2022-02-24 01:31:04 -08:00
chrislu
63062ed7f0 mount2: fix unlink 2022-02-18 01:10:53 -08:00
chrislu
e8ce30fdc5 mount2: adjust file mode 2022-02-18 00:47:02 -08:00
chrislu
f9d33f70b0 return fuse.Status when looking up by inode 2022-02-18 00:45:43 -08:00
chrislu
6ac066d1dc count lookup or not 2022-02-16 16:49:03 -08:00
chrislu
a6bc67c34c less logs 2022-02-16 08:38:51 -08:00
chrislu
dbeeda8123 listen for metadata updates 2022-02-14 01:09:31 -08:00
chrislu
24290fed9d add comments 2022-02-13 06:02:21 -08:00
chrislu
6a921e15f3 forget() factor in nlookup 2022-02-13 05:49:29 -08:00
chrislu
a1ef0e48a9 doc 2022-02-13 04:22:02 -08:00
chrislu
6a42cb6b0b supports mknod, unlink 2022-02-13 03:09:24 -08:00