gsh-digital-services/seaweedFS
5
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

mount: stream all filer mutations over single ordered gRPC stream (#8770)

Browse Source 
* filer: add StreamMutateEntry bidi streaming RPC

Add a bidirectional streaming RPC that carries all filer mutation
types (create, update, delete, rename) over a single ordered stream.
This eliminates per-request connection overhead for pipelined
operations and guarantees mutation ordering within a stream.

The server handler delegates each request to the existing unary
handlers (CreateEntry, UpdateEntry, DeleteEntry) and uses a proxy
stream adapter for rename operations to reuse StreamRenameEntry logic.

The is_last field signals completion for multi-response operations
(rename sends multiple events per request; create/update/delete
always send exactly one response with is_last=true).

* mount: add streaming mutation multiplexer (streamMutateMux)

Implement a client-side multiplexer that routes all filer mutation RPCs
(create, update, delete, rename) over a single bidirectional gRPC
stream. Multiple goroutines submit requests through a send channel;
a dedicated sendLoop serializes them on the stream; a recvLoop
dispatches responses to waiting callers via per-request channels.

Key features:
- Lazy stream opening on first use
- Automatic reconnection on stream failure
- Permanent fallback to unary RPCs if filer returns Unimplemented
- Monotonic request_id for response correlation
- Multi-response support for rename operations (is_last signaling)

The mux is initialized on WFS and closed during unmount cleanup.
No call sites use it yet — wiring comes in subsequent commits.

* mount: route CreateEntry and UpdateEntry through streaming mux

Wire all CreateEntry call sites to use wfs.streamCreateEntry() which
routes through the StreamMutateEntry stream when available, falling
back to unary RPCs otherwise. Also wire Link's UpdateEntry calls
through wfs.streamUpdateEntry().

Updated call sites:
- flushMetadataToFiler (file flush after write)
- Mkdir (directory creation)
- Symlink (symbolic link creation)
- createRegularFile non-deferred path (Mknod)
- flushFileMetadata (periodic metadata flush)
- Link (hard link: update source + create link + rollback)

* mount: route UpdateEntry and DeleteEntry through streaming mux

Wire remaining mutation call sites through the streaming mux:

- saveEntry (Setattr/chmod/chown/utimes) → streamUpdateEntry
- Unlink → streamDeleteEntry (replaces RemoveWithResponse)
- Rmdir → streamDeleteEntry (replaces RemoveWithResponse)

All filer mutations except Rename now go through StreamMutateEntry
when the filer supports it, with automatic unary RPC fallback.

* mount: route Rename through streaming mux

Wire Rename to use streamMutate.Rename() when available, with
fallback to the existing StreamRenameEntry unary stream.

The streaming mux sends rename as a StreamRenameEntryRequest oneof
variant. The server processes it through the existing rename logic
and sends multiple StreamRenameEntryResponse events (one per moved
entry), with is_last=true on the final response.

All filer mutations now go through a single ordered stream.

* mount: fix stream mux connection ownership

WithGrpcClient(streamingMode=true) closes the gRPC connection when
the callback returns, destroying the stream. Own the connection
directly via pb.GrpcDial so it stays alive for the stream's lifetime.
Close it explicitly in recvLoop on stream failure and in Close on
shutdown.

* mount: fix rename failure for deferred-create files

Three fixes for rename operations over the streaming mux:

1. lookupEntry: fall back to local metadata store when filer returns
   "not found" for entries in uncached directories. Files created with
   deferFilerCreate=true exist only in the local leveldb store until
   flushed; lookupEntry skipped the local store when the parent
   directory had never been readdir'd, causing rename to fail with
   ENOENT.

2. Rename: wait for pending async flushes and force synchronous flush
   of dirty metadata before sending rename to the filer. Covers the
   writebackCache case where close() defers the flush to a background
   worker that may not complete before rename fires.

3. StreamMutateEntry: propagate rename errors from server to client.
   Add error/errno fields to StreamMutateEntryResponse so the mount
   can map filer errors to correct FUSE status codes instead of
   silently returning OK. Also fix the existing Rename error handler
   which could return fuse.OK on unrecognized errors.

* mount: fix streaming mux error handling, sendLoop lifecycle, and fallback

Address PR review comments:

1. Server: populate top-level Error/Errno on StreamMutateEntryResponse for
   create/update/delete errors, not just rename. Previously update errors
   were silently dropped and create/delete errors were only in nested
   response fields that the client didn't check.

2. Client: check nested error fields in CreateEntry (ErrorCode, Error)
   and DeleteEntry (Error) responses, matching CreateEntryWithResponse
   behavior.

3. Fix sendLoop lifecycle: give each stream generation a stopSend channel.
   recvLoop closes it on error to stop the paired sendLoop. Previously a
   reconnect left the old sendLoop draining sendCh, breaking ordering.

4. Transparent fallback: stream helpers and doRename fall back to unary
   RPCs on transport errors (ErrStreamTransport), including the first
   Unimplemented from ensureStream. Previously the first call failed
   instead of degrading.

5. Filer rotation in openStream: try all filer addresses on dial failure,
   matching WithFilerClient behavior. Stop early on Unimplemented.

6. Pass metadata-bearing context to StreamMutateEntry RPC call so
   sw-client-id header is actually sent.

7. Gate lookupEntry local-cache fallback on open dirty handle or pending
   async flush to avoid resurrecting deleted/renamed entries.

8. Remove dead code in flushFileMetadata (err=nil followed by if err!=nil).

9. Use string matching for rename error-to-errno mapping in the mount to
   stay portable across Linux/macOS (numeric errno values differ).

* mount: make failAllPending idempotent with delete-before-close

Change failAllPending to collect pending entries into a local slice
(deleting from the sync.Map first) before closing channels. This
prevents double-close panics if called concurrently. Also remove the
unused err parameter.

* mount: add stream generation tracking and teardownStream

Introduce a generation counter on streamMutateMux that increments each
time a new stream is created. Requests carry the generation they were
enqueued for so sendLoop can reject stale requests after reconnect.

Add teardownStream(gen) which is idempotent (only acts when gen matches
current generation and stream is non-nil). Both sendLoop and recvLoop
call it on error, replacing the inline cleanup in recvLoop. sendLoop now
actively triggers teardown on send errors instead of silently exiting.

ensureStream waits for the prior generation's recvDone before creating a
new stream, ensuring all old pending waiters are failed before reconnect.
recvLoop now takes the stream, generation, and recvDone channel as
parameters to avoid accessing shared fields without the lock.

* mount: harden Close to prevent races with teardownStream

Nil out stream, cancel, and grpcConn under the lock so that any
concurrent teardownStream call from recvLoop/sendLoop becomes a no-op.
Call failAllPending before closing sendCh to unblock waiters promptly.
Guard recvDone with a nil check for the case where Close is called
before any stream was ever opened.

* mount: make errCh receive ctx-aware in doUnary and Rename

Replace the blocking <-sendReq.errCh with a select that also observes
ctx.Done(). If sendLoop exits via stopSend without consuming a buffered
request, the caller now returns ctx.Err() instead of blocking forever.
The buffered errCh (capacity 1) ensures late acknowledgements from
sendLoop don't block the sender.

* mount: fix sendLoop/Close race and recvLoop/teardown pending channel race

Three related fixes:

1. Stop closing sendCh in Close(). Closing the shared producer channel
   races with callers who passed ensureStream() but haven't sent yet,
   causing send-on-closed-channel panics. sendCh is now left open;
   ensureStream checks m.closed to reject new callers.

2. Drain buffered sendCh items on shutdown. sendLoop defers drainSendCh()
   on exit so buffered requests get an ErrStreamTransport on their errCh
   instead of blocking forever. Close() drains again for any stragglers
   enqueued between sendLoop's drain and the final shutdown.

3. Move failAllPending from teardownStream into recvLoop's defer.
   teardownStream (called from sendLoop on send error) was closing
   pending response channels while recvLoop could be between
   pending.Load and the channel send — a send-on-closed-channel panic.
   recvLoop is now the sole closer of pending channels, eliminating
   the race. Close() waits on recvDone (with cancel() to guarantee
   Recv unblocks) so pending cleanup always completes.

* filer/mount: add debug logging for hardlink lifecycle

Add V(0) logging at every point where a HardLinkId is created, stored,
read, or deleted to trace orphaned hardlink references. Logging covers:

- gRPC server: CreateEntry/UpdateEntry when request carries HardLinkId
- FilerStoreWrapper: InsertEntry/UpdateEntry when entry has HardLinkId
- handleUpdateToHardLinks: entry path, HardLinkId, counter, chunk count
- setHardLink: KvPut with blob size
- maybeReadHardLink: V(1) on read attempt and successful decode
- DeleteHardLink: counter decrement/deletion events
- Mount Link(): when NewHardLinkId is generated and link is created

This helps diagnose how a git pack .rev file ended up with a
HardLinkId during a clone (no hard links should be involved).

* test: add git clone/pull integration test for FUSE mount

Shell script that exercises git operations on a SeaweedFS mount:
1. Creates a bare repo on the mount
2. Clones locally, makes 3 commits, pushes to mount
3. Clones from mount bare repo into an on-mount working dir
4. Verifies clone integrity (files, content, commit hashes)
5. Pushes 2 more commits with renames and deletes
6. Checks out an older revision on the mount clone
7. Returns to branch and pulls with real changes
8. Verifies file content, renames, deletes after pull
9. Checks git log integrity and clean status

27 assertions covering file existence, content, commit hashes,
file counts, renames, deletes, and git status. Run against any
existing mount: bash test-git-on-mount.sh /path/to/mount

* test: add git clone/pull FUSE integration test to CI suite

Add TestGitOperations to the existing fuse_integration test framework.
The test exercises git's full file operation surface on the mount:

  1. Creates a bare repo on the mount (acts as remote)
  2. Clones locally, makes 3 commits (files, bulk data, renames), pushes
  3. Clones from mount bare repo into an on-mount working dir
  4. Verifies clone integrity (content, commit hash, file count)
  5. Pushes 2 more commits with new files, renames, and deletes
  6. Checks out an older revision on the mount clone
  7. Returns to branch and pulls with real fast-forward changes
  8. Verifies post-pull state: content, renames, deletes, file counts
  9. Checks git log integrity (5 commits) and clean status

Runs automatically in the existing fuse-integration.yml CI workflow.

* mount: fix permission check with uid/gid mapping

The permission checks in createRegularFile() and Access() compared
the caller's local uid/gid against the entry's filer-side uid/gid
without applying the uid/gid mapper. With -map.uid 501:0, a directory
created as uid 0 on the filer would not match the local caller uid
501, causing hasAccess() to fall through to "other" permission bits
and reject write access (0755 → other has r-x, no w).

Fix: map entry uid/gid from filer-space to local-space before the
hasAccess() call so both sides are in the same namespace.

This fixes rsync -a failing with "Permission denied" on mkstempat
when using uid/gid mapping.

* mount: fix Mkdir/Symlink returning filer-side uid/gid to kernel

Mkdir and Symlink used `defer wfs.mapPbIdFromFilerToLocal(entry)` to
restore local uid/gid, but `outputPbEntry` writes the kernel response
before the function returns — so the kernel received filer-side
uid/gid (e.g., 0:0). macFUSE then caches these and rejects subsequent
child operations (mkdir, create) because the caller uid (501) doesn't
match the directory owner (0), and "other" bits (0755 → r-x) lack
write permission.

Fix: replace the defer with an explicit call to mapPbIdFromFilerToLocal
before outputPbEntry, so the kernel gets local uid/gid.

Also add nil guards for UidGidMapper in Access and createRegularFile
to prevent panics in tests that don't configure a mapper.

This fixes rsync -a "Permission denied" on mkpathat for nested
directories when using uid/gid mapping.

* mount: fix Link outputting filer-side uid/gid to kernel, add nil guards

Link had the same defer-before-outputPbEntry bug as Mkdir and Symlink:
the kernel received filer-side uid/gid because the defer hadn't run
yet when outputPbEntry wrote the response.

Also add nil guards for UidGidMapper in Access and createRegularFile
so tests without a mapper don't panic.

Audit of all outputPbEntry/outputFilerEntry call sites:
- Mkdir: fixed in prior commit (explicit map before output)
- Symlink: fixed in prior commit (explicit map before output)
- Link: fixed here (explicit map before output)
- Create (existing file): entry from maybeLoadEntry (already mapped)
- Create (deferred): entry has local uid/gid (never mapped to filer)
- Create (non-deferred): createRegularFile defer runs before return
- Mknod: createRegularFile defer runs before return
- Lookup: entry from lookupEntry (already mapped)
- GetAttr: entry from maybeReadEntry/maybeLoadEntry (already mapped)
- readdir: entry from cache (mapIdFromFilerToLocal) or filer (mapped)
- saveEntry: no kernel output
- flushMetadataToFiler: no kernel output
- flushFileMetadata: no kernel output

* test: fix git test for same-filesystem FUSE clone

When both the bare repo and working clone live on the same FUSE mount,
git's local transport uses hardlinks and cross-repo stat calls that
fail on FUSE. Fix:

- Use --no-local on clone to disable local transport optimizations
- Use reset --hard instead of checkout to stay on branch
- Use fetch + reset --hard origin/<branch> instead of git pull to
  avoid local transport stat failures during fetch

* adjust logging

* test: use plain git clone/pull to exercise real FUSE behavior

Remove --no-local and fetch+reset workarounds. The test should use
the same git commands users run (clone, reset --hard, pull) so it
reveals real FUSE issues rather than hiding them.

* test: enable V(1) logging for filer/mount and collect logs on failure

- Run filer and mount with -v=1 so hardlink lifecycle logs (V(0):
  create/delete/insert, V(1): read attempts) are captured
- On test failure, automatically dump last 16KB of all process logs
  (master, volume, filer, mount) to test output
- Copy process logs to /tmp/seaweedfs-fuse-logs/ for CI artifact upload
- Update CI workflow to upload SeaweedFS process logs alongside test output

* mount: clone entry for filer flush to prevent uid/gid race

flushMetadataToFiler and flushFileMetadata used entry.GetEntry() which
returns the file handle's live proto entry pointer, then mutated it
in-place via mapPbIdFromLocalToFiler. During the gRPC call window, a
concurrent Lookup (which takes entryLock.RLock but NOT fhLockTable)
could observe filer-side uid/gid (e.g., 0:0) on the file handle entry
and return it to the kernel. The kernel caches these attributes, so
subsequent opens by the local user (uid 501) fail with EACCES.

Fix: proto.Clone the entry before mapping uid/gid for the filer request.
The file handle's live entry is never mutated, so concurrent Lookup
always sees local uid/gid.

This fixes the intermittent "Permission denied" on .git/FETCH_HEAD
after the first git pull on a mount with uid/gid mapping.

* mount: add debug logging for stale lock file investigation

Add V(0) logging to trace the HEAD.lock recreation issue:
- Create: log when O_EXCL fails (file already exists) with uid/gid/mode
- completeAsyncFlush: log resolved path, saved path, dirtyMetadata,
  isDeleted at entry to trace whether async flush fires after rename
- flushMetadataToFiler: log the dir/name/fullpath being flushed

This will show whether the async flush is recreating the lock file
after git renames HEAD.lock → HEAD.

* mount: prevent async flush from recreating renamed .lock files

When git renames HEAD.lock → HEAD, the async flush from the prior
close() can run AFTER the rename and re-insert HEAD.lock into the
meta cache via its CreateEntryRequest response event. The next git
pull then sees HEAD.lock and fails with "File exists".

Fix: add isRenamed flag on FileHandle, set by Rename before waiting
for the pending async flush. The async flush checks this flag and
skips the metadata flush for renamed files (same pattern as isDeleted
for unlinked files). The data pages still flush normally.

The Rename handler flushes deferred metadata synchronously (Case 1)
before setting isRenamed, ensuring the entry exists on the filer for
the rename to proceed. For already-released handles (Case 2), the
entry was created by a prior flush.

* mount: also mark renamed inodes via entry.Attributes.Inode fallback

When GetInode fails (Forget already removed the inode mapping), the
Rename handler couldn't find the pending async flush to set isRenamed.
The async flush then recreated the .lock file on the filer.

Fix: fall back to oldEntry.Attributes.Inode to find the pending async
flush when the inode-to-path mapping is gone. Also extract
MarkInodeRenamed into a method on FileHandleToInode for clarity.

* mount: skip async metadata flush when saved path no longer maps to inode

The isRenamed flag approach failed for refs/remotes/origin/HEAD.lock
because neither GetInode nor oldEntry.Attributes.Inode could find the
inode (Forget already evicted the mapping, and the entry's stored
inode was 0).

Add a direct check in completeAsyncFlush: before flushing metadata,
verify that the saved path still maps to this inode in the inode-to-path
table. If the path was renamed or removed (inode mismatch or not found),
skip the metadata flush to avoid recreating a stale entry.

This catches all rename cases regardless of whether the Rename handler
could set the isRenamed flag.

* mount: wait for pending async flush in Unlink before filer delete

Unlink was deleting the filer entry first, then marking the draining
async-flush handle as deleted. The async flush worker could race between
these two operations and recreate the just-unlinked entry on the filer.
This caused git's .lock files (e.g. refs/remotes/origin/HEAD.lock) to
persist after git pull, breaking subsequent git operations.

Move the isDeleted marking and add waitForPendingAsyncFlush() before
the filer delete so any in-flight flush completes first. Even if the
worker raced past the isDeleted check, the wait ensures it finishes
before the filer delete cleans up any recreated entry.

* mount: reduce async flush and metadata flush log verbosity

Raise completeAsyncFlush entry log, saved-path-mismatch skip log, and
flushMetadataToFiler entry log from V(0) to V(3)/V(4). These fire for
every file close with writebackCache and are too noisy for normal use.

* filer: reduce hardlink debug log verbosity from V(0) to V(4)

HardLinkId logs in filerstore_wrapper, filerstore_hardlink, and
filer_grpc_server fire on every hardlinked file operation (git pack
files use hardlinks extensively) and produce excessive noise.

* mount/filer: reduce noisy V(0) logs for link, rmdir, and empty folder check

- weedfs_link.go: hardlink creation logs V(0) → V(4)
- weedfs_dir_mkrm.go: non-empty folder rmdir error V(0) → V(1)
- empty_folder_cleaner.go: "not empty" check log V(0) → V(4)

* filer: handle missing hardlink KV as expected, not error

A "kv: not found" on hardlink read is normal when the link blob was
already cleaned up but a stale entry still references it. Log at V(1)
for not-found; keep Error level for actual KV failures.

* test: add waitForDir before git pull in FUSE git operations test

After git reset --hard, the FUSE mount's metadata cache may need a
moment to settle on slow CI. The git pull subprocess (unpack-objects)
could fail to stat the working directory. Poll for up to 5s.

* Update git_operations_test.go

* wait

* test: simplify FUSE test framework to use weed mini

Replace the 4-process setup (master + volume + filer + mount) with
2 processes: "weed mini" (all-in-one) + "weed mount". This simplifies
startup, reduces port allocation, and is faster on CI.

* test: fix mini flag -admin → -admin.ui
This commit is contained in:
Chris Lu
2026-03-25 20:06:34 -07:00
committed by GitHub
parent 29bdbb3c48
commit 94bfa2b340
30 changed files with 2342 additions and 480 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
weed/mount/filehandle.go
View File

@@ -36,6 +36,7 @@ type FileHandle struct {
savedName string // last known file name if inode-to-path state is forgotten
isDeleted bool
isRenamed bool // set by Rename before waiting for async flush; skips old-path metadata flush
// RDMA chunk offset cache for performance optimization
chunkOffsetCache []int64

11
weed/mount/filehandle_map.go
View File

@@ -38,6 +38,17 @@ func (i *FileHandleToInode) FindFileHandle(inode uint64) (fh *FileHandle, found
return
}
// MarkInodeRenamed sets isRenamed on any file handle associated with the
// given inode. This prevents the async flush from recreating a renamed
// file's metadata under its old path.
func (i *FileHandleToInode) MarkInodeRenamed(inode uint64) {
i.RLock()
defer i.RUnlock()
if fh, ok := i.inode2fh[inode]; ok {
fh.isRenamed = true
}
}
func (i *FileHandleToInode) AcquireFileHandle(wfs *WFS, inode uint64, entry *filer_pb.Entry) *FileHandle {
i.Lock()
defer i.Unlock()

27
weed/mount/weedfs.go
View File

@@ -133,6 +133,11 @@ type WFS struct {
// the same inode, preventing stale metadata from overwriting the async flush.
pendingAsyncFlushMu sync.Mutex
pendingAsyncFlush map[uint64]chan struct{}
// streamMutate is the multiplexed streaming gRPC connection for all filer
// mutations (create, update, delete, rename). All mutations go through one
// ordered stream to prevent cross-operation reordering.
streamMutate *streamMutateMux
}
const (
@@ -289,6 +294,7 @@ func NewSeaweedFileSystem(option *Option) *WFS {
}
wfs.startAsyncFlushWorkers(numWorkers)
}
wfs.streamMutate = newStreamMutateMux(wfs)
wfs.copyBufferPool.New = func() any {
return make([]byte, option.ChunkSizeLimit)
}
@@ -404,6 +410,27 @@ func (wfs *WFS) lookupEntry(fullpath util.FullPath) (*filer.Entry, fuse.Status)
entry, err := filer_pb.GetEntry(context.Background(), wfs, fullpath)
if err != nil {
if err == filer_pb.ErrNotFound {
// The entry may exist in the local store from a deferred create
// (deferFilerCreate=true) that hasn't been flushed yet. Only trust
// the local store when an open file handle or pending async flush
// confirms the entry is genuinely local-only; otherwise a stale
// cache hit could resurrect a deleted/renamed entry.
if inode, inodeFound := wfs.inodeToPath.GetInode(fullpath); inodeFound {
hasDirtyHandle := false
if fh, fhFound := wfs.fhMap.FindFileHandle(inode); fhFound && fh.dirtyMetadata {
hasDirtyHandle = true
}
wfs.pendingAsyncFlushMu.Lock()
_, hasPendingFlush := wfs.pendingAsyncFlush[inode]
wfs.pendingAsyncFlushMu.Unlock()
if hasDirtyHandle || hasPendingFlush {
if localEntry, localErr := wfs.metaCache.FindEntry(context.Background(), fullpath); localErr == nil && localEntry != nil {
glog.V(4).Infof("lookupEntry found deferred entry in local cache %s", fullpath)
return localEntry, fuse.OK
}
}
}
glog.V(4).Infof("lookupEntry not found %s", fullpath)
return nil, fuse.ENOENT
}

8
weed/mount/weedfs_access.go
View File

@@ -40,7 +40,13 @@ func (wfs *WFS) Access(cancel <-chan struct{}, input *fuse.AccessIn) (code fuse.
if entry == nil || entry.Attributes == nil {
return fuse.EIO
}
if hasAccess(input.Uid, input.Gid, entry.Attributes.Uid, entry.Attributes.Gid, entry.Attributes.FileMode, input.Mask) {
// Map entry uid/gid from filer-space to local-space so the permission
// check compares like with like (caller uid/gid from FUSE are local).
fileUid, fileGid := entry.Attributes.Uid, entry.Attributes.Gid
if wfs.option.UidGidMapper != nil {
fileUid, fileGid = wfs.option.UidGidMapper.FilerToLocal(fileUid, fileGid)
}
if hasAccess(input.Uid, input.Gid, fileUid, fileGid, entry.Attributes.FileMode, input.Mask) {
return fuse.OK
}
return fuse.EACCES

21
weed/mount/weedfs_async_flush.go
View File

@@ -51,6 +51,9 @@ func (wfs *WFS) processAsyncFlushItem(item *asyncFlushItem) {
// This enables close() to return immediately for small file workloads (e.g., rsync),
// while the actual I/O happens concurrently in the background.
func (wfs *WFS) completeAsyncFlush(fh *FileHandle) {
glog.V(4).Infof("completeAsyncFlush inode %d fh %d saved=%s/%s dirtyMetadata=%v isDeleted=%v isRenamed=%v",
fh.inode, fh.fh, fh.savedDir, fh.savedName, fh.dirtyMetadata, fh.isDeleted, fh.isRenamed)
// Phase 1: Flush dirty pages — seals writable chunks, uploads to volume servers, and waits.
// The underlying UploadWithRetry already retries transient HTTP/gRPC errors internally,
// so a failure here indicates a persistent issue; the chunk data has been freed.
@@ -65,8 +68,19 @@ func (wfs *WFS) completeAsyncFlush(fh *FileHandle) {
// handle. In that case the filer entry is already gone and
// flushing would recreate it. The uploaded chunks become orphans
// and are cleaned up by volume.fsck.
if fh.isDeleted {
glog.V(3).Infof("completeAsyncFlush inode %d: file was unlinked, skipping metadata flush", fh.inode)
if fh.isDeleted || fh.isRenamed {
if fh.isDeleted {
glog.V(3).Infof("completeAsyncFlush inode %d: file was unlinked, skipping metadata flush", fh.inode)
} else {
glog.V(3).Infof("completeAsyncFlush inode %d: file was renamed, skipping old-path metadata flush (Rename handles it)", fh.inode)
}
} else if savedInode, found := wfs.inodeToPath.GetInode(util.FullPath(fh.savedDir).Child(fh.savedName)); !found || savedInode != fh.inode {
// The saved path no longer maps to this inode — the file was
// renamed (or deleted and recreated). Flushing metadata under
// the old path would re-insert a stale entry into the meta
// cache, breaking git's lock file protocol.
glog.V(3).Infof("completeAsyncFlush inode %d: saved path %s/%s no longer maps to this inode, skipping metadata flush",
fh.inode, fh.savedDir, fh.savedName)
} else {
// Resolve the current path for metadata flush.
//
@@ -121,4 +135,7 @@ func (wfs *WFS) WaitForAsyncFlush() {
if wfs.asyncFlushCh != nil {
close(wfs.asyncFlushCh)
}
if wfs.streamMutate != nil {
wfs.streamMutate.Close()
}
}

53
weed/mount/weedfs_dir_mkrm.go
View File

@@ -49,25 +49,24 @@ func (wfs *WFS) Mkdir(cancel <-chan struct{}, in *fuse.MkdirIn, name string, out
entryFullPath := dirFullPath.Child(name)
err := wfs.WithFilerClient(false, func(client filer_pb.SeaweedFilerClient) error {
wfs.mapPbIdFromLocalToFiler(newEntry)
// Defer restoring to local uid/gid AFTER the entry is sent to the filer
// but BEFORE outputPbEntry writes attributes to the kernel. We restore
// explicitly below instead of using defer so the kernel gets local values.
wfs.mapPbIdFromLocalToFiler(newEntry)
defer wfs.mapPbIdFromFilerToLocal(newEntry)
request := &filer_pb.CreateEntryRequest{
Directory: string(dirFullPath),
Entry: newEntry,
Signatures: []int32{wfs.signature},
SkipCheckParentDirectory: true,
}
glog.V(1).Infof("mkdir: %v", request)
resp, err := filer_pb.CreateEntryWithResponse(context.Background(), client, request)
if err != nil {
glog.V(0).Infof("mkdir %s: %v", entryFullPath, err)
return err
}
request := &filer_pb.CreateEntryRequest{
Directory: string(dirFullPath),
Entry: newEntry,
Signatures: []int32{wfs.signature},
SkipCheckParentDirectory: true,
}
glog.V(1).Infof("mkdir: %v", request)
resp, err := wfs.streamCreateEntry(context.Background(), request)
if err != nil {
glog.V(0).Infof("mkdir %s: %v", entryFullPath, err)
} else {
event := resp.GetMetadataEvent()
if event == nil {
event = metadataCreateEvent(string(dirFullPath), newEntry)
@@ -77,16 +76,20 @@ func (wfs *WFS) Mkdir(cancel <-chan struct{}, in *fuse.MkdirIn, name string, out
wfs.inodeToPath.InvalidateChildrenCache(dirFullPath)
}
wfs.inodeToPath.TouchDirectory(dirFullPath)
return nil
})
}
glog.V(3).Infof("mkdir %s: %v", entryFullPath, err)
if err != nil {
wfs.mapPbIdFromFilerToLocal(newEntry)
return fuse.EIO
}
// Map uid/gid back to local-space before writing attributes to the
// kernel. The kernel (especially macFUSE) caches these and uses them
// for subsequent permission checks on children.
wfs.mapPbIdFromFilerToLocal(newEntry)
inode := wfs.inodeToPath.Lookup(entryFullPath, newEntry.Attributes.Crtime, true, false, 0, true)
wfs.outputPbEntry(out, inode, newEntry)
@@ -112,10 +115,16 @@ func (wfs *WFS) Rmdir(cancel <-chan struct{}, header *fuse.InHeader, name string
entryFullPath := dirFullPath.Child(name)
glog.V(3).Infof("remove directory: %v", entryFullPath)
ignoreRecursiveErr := true // ignore recursion error since the OS should manage it
resp, err := filer_pb.RemoveWithResponse(context.Background(), wfs, string(dirFullPath), name, true, false, ignoreRecursiveErr, false, []int32{wfs.signature})
deleteReq := &filer_pb.DeleteEntryRequest{
Directory: string(dirFullPath),
Name: name,
IsDeleteData: true,
IgnoreRecursiveError: true, // ignore recursion error since the OS should manage it
Signatures: []int32{wfs.signature},
}
resp, err := wfs.streamDeleteEntry(context.Background(), deleteReq)
if err != nil {
glog.V(0).Infof("remove %s: %v", entryFullPath, err)
glog.V(1).Infof("remove %s: %v", entryFullPath, err)
if strings.Contains(err.Error(), filer.MsgFailDelNonEmptyFolder) {
return fuse.Status(syscall.ENOTEMPTY)
}

80
weed/mount/weedfs_file_mkrm.go
View File

@@ -41,6 +41,8 @@ func (wfs *WFS) Create(cancel <-chan struct{}, in *fuse.CreateIn, name string, o
return fuse.EIO
}
if in.Flags&syscall.O_EXCL != 0 {
glog.V(0).Infof("Create O_EXCL %s: already exists (uid=%d gid=%d mode=%o)",
entryFullPath, newEntry.Attributes.Uid, newEntry.Attributes.Gid, newEntry.Attributes.FileMode)
return fuse.Status(syscall.EEXIST)
}
inode = wfs.inodeToPath.Lookup(entryFullPath, newEntry.Attributes.Crtime, false, len(newEntry.HardLinkId) > 0, newEntry.Attributes.Inode, true)
@@ -170,11 +172,37 @@ func (wfs *WFS) Unlink(cancel <-chan struct{}, header *fuse.InHeader, name strin
return fuse.EPERM
}
// Before deleting from the filer, mark any draining async-flush handle
// as deleted and wait for it to complete. Without this, the async flush
// can race with the filer delete and recreate the just-unlinked entry
// (the worker checks isDeleted, but it may have already passed that check
// before Unlink sets the flag). By waiting here, any in-flight flush
// finishes first; even if it recreated the entry, the filer delete below
// will remove it again.
if inode, found := wfs.inodeToPath.GetInode(entryFullPath); found {
if fh, fhFound := wfs.fhMap.FindFileHandle(inode); fhFound {
fh.isDeleted = true
}
wfs.waitForPendingAsyncFlush(inode)
} else if entry != nil && entry.Attributes != nil && entry.Attributes.Inode != 0 {
inodeFromEntry := entry.Attributes.Inode
if fh, fhFound := wfs.fhMap.FindFileHandle(inodeFromEntry); fhFound {
fh.isDeleted = true
}
wfs.waitForPendingAsyncFlush(inodeFromEntry)
}
// first, ensure the filer store can correctly delete
glog.V(3).Infof("remove file: %v", entryFullPath)
// Always let the filer decide whether to delete chunks based on its authoritative data.
// The filer has the correct hard link count and will only delete chunks when appropriate.
resp, err := filer_pb.RemoveWithResponse(context.Background(), wfs, string(dirFullPath), name, true, false, false, false, []int32{wfs.signature})
deleteReq := &filer_pb.DeleteEntryRequest{
Directory: string(dirFullPath),
Name: name,
IsDeleteData: true,
Signatures: []int32{wfs.signature},
}
resp, err := wfs.streamDeleteEntry(context.Background(), deleteReq)
if err != nil {
glog.V(0).Infof("remove %s: %v", entryFullPath, err)
return fuse.OK
@@ -192,15 +220,6 @@ func (wfs *WFS) Unlink(cancel <-chan struct{}, header *fuse.InHeader, name strin
}
wfs.inodeToPath.TouchDirectory(dirFullPath)
// If there is an async-draining handle for this file, mark it as deleted
// so the background flush skips the metadata write instead of recreating
// the just-unlinked entry. The handle is still in fhMap during drain.
if inode, found := wfs.inodeToPath.GetInode(entryFullPath); found {
if fh, fhFound := wfs.fhMap.FindFileHandle(inode); fhFound {
fh.isDeleted = true
}
}
wfs.inodeToPath.RemovePath(entryFullPath)
return fuse.OK
@@ -220,7 +239,13 @@ func (wfs *WFS) createRegularFile(dirFullPath util.FullPath, name string, mode u
if parentEntry == nil || parentEntry.Attributes == nil {
return 0, nil, fuse.EIO
}
if !hasAccess(uid, gid, parentEntry.Attributes.Uid, parentEntry.Attributes.Gid, parentEntry.Attributes.FileMode, fuse.W_OK|fuse.X_OK) {
// Map parent dir uid/gid from filer-space to local-space so the
// permission check compares like with like (caller uid/gid are local).
parentUid, parentGid := parentEntry.Attributes.Uid, parentEntry.Attributes.Gid
if wfs.option.UidGidMapper != nil {
parentUid, parentGid = wfs.option.UidGidMapper.FilerToLocal(parentUid, parentGid)
}
if !hasAccess(uid, gid, parentUid, parentGid, parentEntry.Attributes.FileMode, fuse.W_OK|fuse.X_OK) {
return 0, nil, fuse.Status(syscall.EACCES)
}
@@ -264,24 +289,21 @@ func (wfs *WFS) createRegularFile(dirFullPath util.FullPath, name string, mode u
return inode, newEntry, fuse.OK
}
err := wfs.WithFilerClient(false, func(client filer_pb.SeaweedFilerClient) error {
wfs.mapPbIdFromLocalToFiler(newEntry)
defer wfs.mapPbIdFromFilerToLocal(newEntry)
wfs.mapPbIdFromLocalToFiler(newEntry)
defer wfs.mapPbIdFromFilerToLocal(newEntry)
request := &filer_pb.CreateEntryRequest{
Directory: string(dirFullPath),
Entry: newEntry,
Signatures: []int32{wfs.signature},
SkipCheckParentDirectory: true,
}
glog.V(1).Infof("createFile: %v", request)
resp, err := filer_pb.CreateEntryWithResponse(context.Background(), client, request)
if err != nil {
glog.V(0).Infof("createFile %s: %v", entryFullPath, err)
return err
}
request := &filer_pb.CreateEntryRequest{
Directory: string(dirFullPath),
Entry: newEntry,
Signatures: []int32{wfs.signature},
SkipCheckParentDirectory: true,
}
glog.V(1).Infof("createFile: %v", request)
resp, err := wfs.streamCreateEntry(context.Background(), request)
if err != nil {
glog.V(0).Infof("createFile %s: %v", entryFullPath, err)
} else {
event := resp.GetMetadataEvent()
if event == nil {
event = metadataCreateEvent(string(dirFullPath), newEntry)
@@ -291,9 +313,7 @@ func (wfs *WFS) createRegularFile(dirFullPath util.FullPath, name string, mode u
wfs.inodeToPath.InvalidateChildrenCache(dirFullPath)
}
wfs.inodeToPath.TouchDirectory(dirFullPath)
return nil
})
}
glog.V(3).Infof("createFile %s: %v", entryFullPath, err)

91
weed/mount/weedfs_file_sync.go
View File

@@ -11,6 +11,7 @@ import (
"github.com/seaweedfs/seaweedfs/weed/glog"
"github.com/seaweedfs/seaweedfs/weed/pb/filer_pb"
"github.com/seaweedfs/seaweedfs/weed/util"
"google.golang.org/protobuf/proto"
)
/**
@@ -168,65 +169,65 @@ func (wfs *WFS) doFlush(fh *FileHandle, uid, gid uint32, allowAsync bool) fuse.S
// This is shared between the synchronous doFlush path and the async flush completion.
func (wfs *WFS) flushMetadataToFiler(fh *FileHandle, dir, name string, uid, gid uint32) error {
fileFullPath := fh.FullPath()
glog.V(4).Infof("flushMetadataToFiler %s/%s inode %d fh %d", dir, name, fh.inode, fh.fh)
fhActiveLock := fh.wfs.fhLockTable.AcquireLock("doFlush", fh.fh, util.ExclusiveLock)
defer fh.wfs.fhLockTable.ReleaseLock(fh.fh, fhActiveLock)
err := wfs.WithFilerClient(false, func(client filer_pb.SeaweedFilerClient) error {
entry := fh.GetEntry()
entry.Name = name // this flush may be just after a rename operation
entry := fh.GetEntry()
entry.Name = name // this flush may be just after a rename operation
if entry.Attributes != nil {
entry.Attributes.Mime = fh.contentType
if entry.Attributes.Uid == 0 {
entry.Attributes.Uid = uid
}
if entry.Attributes.Gid == 0 {
entry.Attributes.Gid = gid
}
entry.Attributes.Mtime = time.Now().Unix()
if entry.Attributes != nil {
entry.Attributes.Mime = fh.contentType
if entry.Attributes.Uid == 0 {
entry.Attributes.Uid = uid
}
request := &filer_pb.CreateEntryRequest{
Directory: string(dir),
Entry: entry.GetEntry(),
Signatures: []int32{wfs.signature},
SkipCheckParentDirectory: true,
if entry.Attributes.Gid == 0 {
entry.Attributes.Gid = gid
}
entry.Attributes.Mtime = time.Now().Unix()
}
glog.V(4).Infof("%s set chunks: %v", fileFullPath, len(entry.GetChunks()))
glog.V(4).Infof("%s set chunks: %v", fileFullPath, len(entry.GetChunks()))
manifestChunks, nonManifestChunks := filer.SeparateManifestChunks(entry.GetChunks())
manifestChunks, nonManifestChunks := filer.SeparateManifestChunks(entry.GetChunks())
chunks, _ := filer.CompactFileChunks(context.Background(), wfs.LookupFn(), nonManifestChunks)
chunks, manifestErr := filer.MaybeManifestize(wfs.saveDataAsChunk(fileFullPath), chunks)
if manifestErr != nil {
// not good, but should be ok
glog.V(0).Infof("MaybeManifestize: %v", manifestErr)
}
entry.Chunks = append(chunks, manifestChunks...)
chunks, _ := filer.CompactFileChunks(context.Background(), wfs.LookupFn(), nonManifestChunks)
chunks, manifestErr := filer.MaybeManifestize(wfs.saveDataAsChunk(fileFullPath), chunks)
if manifestErr != nil {
// not good, but should be ok
glog.V(0).Infof("MaybeManifestize: %v", manifestErr)
}
entry.Chunks = append(chunks, manifestChunks...)
wfs.mapPbIdFromLocalToFiler(request.Entry)
defer wfs.mapPbIdFromFilerToLocal(request.Entry)
// Clone the proto entry for the filer request so that mapPbIdFromLocalToFiler
// does not mutate the file handle's live entry. Without the clone, a concurrent
// Lookup can observe filer-side uid/gid on the file handle entry and return it
// to the kernel, which caches it and then rejects opens by the local user.
requestEntry := proto.Clone(entry.GetEntry()).(*filer_pb.Entry)
request := &filer_pb.CreateEntryRequest{
Directory: string(dir),
Entry: requestEntry,
Signatures: []int32{wfs.signature},
SkipCheckParentDirectory: true,
}
resp, err := filer_pb.CreateEntryWithResponse(context.Background(), client, request)
if err != nil {
glog.Errorf("fh flush create %s: %v", fileFullPath, err)
return fmt.Errorf("fh flush create %s: %v", fileFullPath, err)
}
wfs.mapPbIdFromLocalToFiler(request.Entry)
event := resp.GetMetadataEvent()
if event == nil {
event = metadataUpdateEvent(string(dir), request.Entry)
}
if applyErr := wfs.applyLocalMetadataEvent(context.Background(), event); applyErr != nil {
glog.Warningf("flush %s: best-effort metadata apply failed: %v", fileFullPath, applyErr)
wfs.inodeToPath.InvalidateChildrenCache(util.FullPath(dir))
}
resp, err := wfs.streamCreateEntry(context.Background(), request)
if err != nil {
glog.Errorf("fh flush create %s: %v", fileFullPath, err)
return fmt.Errorf("fh flush create %s: %v", fileFullPath, err)
}
return nil
})
event := resp.GetMetadataEvent()
if event == nil {
event = metadataUpdateEvent(string(dir), request.Entry)
}
if applyErr := wfs.applyLocalMetadataEvent(context.Background(), event); applyErr != nil {
glog.Warningf("flush %s: best-effort metadata apply failed: %v", fileFullPath, applyErr)
wfs.inodeToPath.InvalidateChildrenCache(util.FullPath(dir))
}
if err == nil {
fh.dirtyMetadata = false

51
weed/mount/weedfs_link.go
View File

@@ -60,8 +60,11 @@ func (wfs *WFS) Link(cancel <-chan struct{}, in *fuse.LinkIn, name string, out *
if len(oldEntry.HardLinkId) == 0 {
oldEntry.HardLinkId = filer.NewHardLinkId()
oldEntry.HardLinkCounter = 1
glog.V(4).Infof("Link: new HardLinkId %x for %s", oldEntry.HardLinkId, oldEntryPath)
}
oldEntry.HardLinkCounter++
glog.V(4).Infof("Link: %s -> %s/%s HardLinkId %x counter=%d",
oldEntryPath, newParentPath, name, oldEntry.HardLinkId, oldEntry.HardLinkCounter)
updateOldEntryRequest := &filer_pb.UpdateEntryRequest{
Directory: oldParentPath,
Entry: oldEntry,
@@ -86,25 +89,23 @@ func (wfs *WFS) Link(cancel <-chan struct{}, in *fuse.LinkIn, name string, out *
}
// apply changes to the filer, and also apply to local metaCache
err := wfs.WithFilerClient(false, func(client filer_pb.SeaweedFilerClient) error {
wfs.mapPbIdFromLocalToFiler(request.Entry)
wfs.mapPbIdFromLocalToFiler(request.Entry)
defer wfs.mapPbIdFromFilerToLocal(request.Entry)
updateResp, err := filer_pb.UpdateEntryWithResponse(context.Background(), client, updateOldEntryRequest)
if err != nil {
return err
}
ctx := context.Background()
updateResp, err := wfs.streamUpdateEntry(ctx, updateOldEntryRequest)
if err == nil {
updateEvent := updateResp.GetMetadataEvent()
if updateEvent == nil {
updateEvent = metadataUpdateEvent(oldParentPath, updateOldEntryRequest.Entry)
}
if applyErr := wfs.applyLocalMetadataEvent(context.Background(), updateEvent); applyErr != nil {
if applyErr := wfs.applyLocalMetadataEvent(ctx, updateEvent); applyErr != nil {
glog.Warningf("link %s: best-effort metadata apply failed: %v", oldEntryPath, applyErr)
wfs.inodeToPath.InvalidateChildrenCache(util.FullPath(oldParentPath))
}
createResp, err := filer_pb.CreateEntryWithResponse(context.Background(), client, request)
}
if err == nil {
var createResp *filer_pb.CreateEntryResponse
createResp, err = wfs.streamCreateEntry(ctx, request)
if err != nil {
// Rollback: restore original HardLinkId/Counter on the source entry
oldEntry.HardLinkId = origHardLinkId
@@ -114,26 +115,26 @@ func (wfs *WFS) Link(cancel <-chan struct{}, in *fuse.LinkIn, name string, out *
Entry: oldEntry,
Signatures: []int32{wfs.signature},
}
if _, rollbackErr := filer_pb.UpdateEntryWithResponse(context.Background(), client, rollbackReq); rollbackErr != nil {
if _, rollbackErr := wfs.streamUpdateEntry(ctx, rollbackReq); rollbackErr != nil {
glog.Warningf("link rollback %s: %v", oldEntryPath, rollbackErr)
}
return err
} else {
createEvent := createResp.GetMetadataEvent()
if createEvent == nil {
createEvent = metadataCreateEvent(string(newParentPath), request.Entry)
}
if applyErr := wfs.applyLocalMetadataEvent(ctx, createEvent); applyErr != nil {
glog.Warningf("link %s: best-effort metadata apply failed: %v", newParentPath.Child(name), applyErr)
wfs.inodeToPath.InvalidateChildrenCache(newParentPath)
}
}
createEvent := createResp.GetMetadataEvent()
if createEvent == nil {
createEvent = metadataCreateEvent(string(newParentPath), request.Entry)
}
if applyErr := wfs.applyLocalMetadataEvent(context.Background(), createEvent); applyErr != nil {
glog.Warningf("link %s: best-effort metadata apply failed: %v", newParentPath.Child(name), applyErr)
wfs.inodeToPath.InvalidateChildrenCache(newParentPath)
}
return nil
})
}
newEntryPath := newParentPath.Child(name)
// Map back to local uid/gid before writing attributes to the kernel.
wfs.mapPbIdFromFilerToLocal(request.Entry)
if err != nil {
glog.V(0).Infof("Link %v -> %s: %v", oldEntryPath, newEntryPath, err)
return fuse.EIO

112
weed/mount/weedfs_metadata_flush.go
View File

@@ -9,6 +9,7 @@ import (
"github.com/seaweedfs/seaweedfs/weed/glog"
"github.com/seaweedfs/seaweedfs/weed/pb/filer_pb"
"github.com/seaweedfs/seaweedfs/weed/util"
"google.golang.org/protobuf/proto"
)
// loopFlushDirtyMetadata periodically flushes dirty file metadata to the filer.
@@ -99,70 +100,65 @@ func (wfs *WFS) flushFileMetadata(fh *FileHandle) error {
glog.V(4).Infof("flushFileMetadata %s fh %d", fileFullPath, fh.fh)
err := wfs.WithFilerClient(false, func(client filer_pb.SeaweedFilerClient) error {
entry := fh.GetEntry()
if entry == nil {
return nil
}
entry.Name = name
if entry.Attributes != nil {
entry.Attributes.Mtime = time.Now().Unix()
}
// Get current chunks - these include chunks that have been uploaded
// but not yet persisted to filer metadata
chunks := entry.GetChunks()
if len(chunks) == 0 {
return nil
}
// Separate manifest and non-manifest chunks
manifestChunks, nonManifestChunks := filer.SeparateManifestChunks(chunks)
// Compact chunks to remove fully overlapped ones
compactedChunks, _ := filer.CompactFileChunks(context.Background(), wfs.LookupFn(), nonManifestChunks)
// Try to create manifest chunks for large files
compactedChunks, manifestErr := filer.MaybeManifestize(wfs.saveDataAsChunk(fileFullPath), compactedChunks)
if manifestErr != nil {
glog.V(0).Infof("flushFileMetadata MaybeManifestize: %v", manifestErr)
}
entry.Chunks = append(compactedChunks, manifestChunks...)
request := &filer_pb.CreateEntryRequest{
Directory: string(dir),
Entry: entry.GetEntry(),
Signatures: []int32{wfs.signature},
SkipCheckParentDirectory: true,
}
wfs.mapPbIdFromLocalToFiler(request.Entry)
defer wfs.mapPbIdFromFilerToLocal(request.Entry)
resp, err := filer_pb.CreateEntryWithResponse(context.Background(), client, request)
if err != nil {
return err
}
event := resp.GetMetadataEvent()
if event == nil {
event = metadataUpdateEvent(string(dir), request.Entry)
}
if applyErr := wfs.applyLocalMetadataEvent(context.Background(), event); applyErr != nil {
glog.Warningf("flushFileMetadata %s: best-effort metadata apply failed: %v", fileFullPath, applyErr)
wfs.inodeToPath.InvalidateChildrenCache(util.FullPath(dir))
}
glog.V(3).Infof("flushed metadata for %s with %d chunks", fileFullPath, len(entry.GetChunks()))
entry := fh.GetEntry()
if entry == nil {
return nil
})
}
entry.Name = name
if entry.Attributes != nil {
entry.Attributes.Mtime = time.Now().Unix()
}
// Get current chunks - these include chunks that have been uploaded
// but not yet persisted to filer metadata
chunks := entry.GetChunks()
if len(chunks) == 0 {
return nil
}
// Separate manifest and non-manifest chunks
manifestChunks, nonManifestChunks := filer.SeparateManifestChunks(chunks)
// Compact chunks to remove fully overlapped ones
compactedChunks, _ := filer.CompactFileChunks(context.Background(), wfs.LookupFn(), nonManifestChunks)
// Try to create manifest chunks for large files
compactedChunks, manifestErr := filer.MaybeManifestize(wfs.saveDataAsChunk(fileFullPath), compactedChunks)
if manifestErr != nil {
glog.V(0).Infof("flushFileMetadata MaybeManifestize: %v", manifestErr)
}
entry.Chunks = append(compactedChunks, manifestChunks...)
// Clone the proto entry so mapPbIdFromLocalToFiler does not mutate the
// file handle's live entry (same race as in flushMetadataToFiler).
requestEntry := proto.Clone(entry.GetEntry()).(*filer_pb.Entry)
request := &filer_pb.CreateEntryRequest{
Directory: string(dir),
Entry: requestEntry,
Signatures: []int32{wfs.signature},
SkipCheckParentDirectory: true,
}
wfs.mapPbIdFromLocalToFiler(request.Entry)
resp, err := wfs.streamCreateEntry(context.Background(), request)
if err != nil {
return err
}
event := resp.GetMetadataEvent()
if event == nil {
event = metadataUpdateEvent(string(dir), request.Entry)
}
if applyErr := wfs.applyLocalMetadataEvent(context.Background(), event); applyErr != nil {
glog.Warningf("flushFileMetadata %s: best-effort metadata apply failed: %v", fileFullPath, applyErr)
wfs.inodeToPath.InvalidateChildrenCache(util.FullPath(dir))
}
glog.V(3).Infof("flushed metadata for %s with %d chunks", fileFullPath, len(entry.GetChunks()))
// Note: We do NOT clear dirtyMetadata here because:
// 1. There may still be dirty pages in the write buffer
// 2. The file may receive more writes before close

128
weed/mount/weedfs_rename.go
View File

@@ -2,6 +2,7 @@ package mount
import (
"context"
"errors"
"fmt"
"io"
"strings"
@@ -14,6 +15,38 @@ import (
"github.com/seaweedfs/seaweedfs/weed/util"
)
// doRename tries the streaming mux first, falling back to unary on transport errors.
func (wfs *WFS) doRename(ctx context.Context, request *filer_pb.StreamRenameEntryRequest, oldPath, newPath util.FullPath) error {
if wfs.streamMutate != nil && wfs.streamMutate.IsAvailable() {
err := wfs.streamMutate.Rename(ctx, request, func(resp *filer_pb.StreamRenameEntryResponse) error {
return wfs.handleRenameResponse(ctx, resp)
})
if err == nil || !errors.Is(err, ErrStreamTransport) {
return err // success or application error
}
glog.V(1).Infof("Rename %s => %s: stream failed, falling back to unary: %v", oldPath, newPath, err)
}
return wfs.WithFilerClient(true, func(client filer_pb.SeaweedFilerClient) error {
stream, streamErr := client.StreamRenameEntry(ctx, request)
if streamErr != nil {
return fmt.Errorf("dir AtomicRenameEntry %s => %s : %v", oldPath, newPath, streamErr)
}
for {
resp, recvErr := stream.Recv()
if recvErr != nil {
if recvErr == io.EOF {
break
}
return fmt.Errorf("dir Rename %s => %s receive: %v", oldPath, newPath, recvErr)
}
if err := wfs.handleRenameResponse(ctx, resp); err != nil {
return err
}
}
return nil
})
}
/** Rename a file
*
* If the target exists it should be atomically replaced. If
@@ -171,53 +204,60 @@ func (wfs *WFS) Rename(cancel <-chan struct{}, in *fuse.RenameIn, oldName string
glog.V(4).Infof("dir Rename %s => %s", oldPath, newPath)
// Ensure the source file's metadata exists on the filer before renaming.
// Two cases can leave the entry only in the local cache:
// 1. deferFilerCreate=true — file handle still open, dirtyMetadata set.
// 2. writebackCache — close() triggered async flush, handle released.
// The filer rename will fail with ENOENT unless we flush/wait first.
if inode, found := wfs.inodeToPath.GetInode(oldPath); found {
// Case 1: handle still open with deferred metadata — flush synchronously
// BEFORE any async flush interference.
if fh, ok := wfs.fhMap.FindFileHandle(inode); ok && fh.dirtyMetadata {
glog.V(4).Infof("dir Rename %s: flushing deferred metadata before rename", oldPath)
if flushStatus := wfs.doFlush(fh, oldEntry.Attributes.Uid, oldEntry.Attributes.Gid, false); flushStatus != fuse.OK {
glog.Warningf("dir Rename %s: flush before rename failed: %v", oldPath, flushStatus)
return flushStatus
}
}
// Case 2: handle already released, async flush may be in flight.
// Mark ALL handles for this inode as renamed so the async flush
// skips old-path metadata creation (which would re-insert the
// renamed entry into the meta cache after rename events clean it up).
wfs.fhMap.MarkInodeRenamed(inode)
wfs.waitForPendingAsyncFlush(inode)
} else if oldEntry != nil && oldEntry.Attributes != nil && oldEntry.Attributes.Inode != 0 {
// GetInode failed (Forget already removed the mapping), but the
// entry's stored inode can still identify pending async flushes.
inode = oldEntry.Attributes.Inode
wfs.fhMap.MarkInodeRenamed(inode)
wfs.waitForPendingAsyncFlush(inode)
}
// update remote filer
err := wfs.WithFilerClient(true, func(client filer_pb.SeaweedFilerClient) error {
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
request := &filer_pb.StreamRenameEntryRequest{
OldDirectory: string(oldDir),
OldName: oldName,
NewDirectory: string(newDir),
NewName: newName,
Signatures: []int32{wfs.signature},
}
request := &filer_pb.StreamRenameEntryRequest{
OldDirectory: string(oldDir),
OldName: oldName,
NewDirectory: string(newDir),
NewName: newName,
Signatures: []int32{wfs.signature},
}
stream, err := client.StreamRenameEntry(ctx, request)
if err != nil {
code = fuse.EIO
return fmt.Errorf("dir AtomicRenameEntry %s => %s : %v", oldPath, newPath, err)
}
for {
resp, recvErr := stream.Recv()
if recvErr != nil {
if recvErr == io.EOF {
break
} else {
if strings.Contains(recvErr.Error(), "not empty") {
code = fuse.Status(syscall.ENOTEMPTY)
} else if strings.Contains(recvErr.Error(), "not directory") {
code = fuse.ENOTDIR
}
return fmt.Errorf("dir Rename %s => %s receive: %v", oldPath, newPath, recvErr)
}
}
if err = wfs.handleRenameResponse(ctx, resp); err != nil {
glog.V(0).Infof("dir Rename %s => %s : %v", oldPath, newPath, err)
return err
}
}
return nil
})
ctx := context.Background()
err := wfs.doRename(ctx, request, oldPath, newPath)
if err != nil {
glog.V(0).Infof("Link: %v", err)
return
glog.V(0).Infof("Rename %s => %s: %v", oldPath, newPath, err)
// Map error strings to FUSE status codes. String matching is used
// instead of raw errno to stay portable across platforms (errno
// numeric values differ between Linux and macOS).
msg := err.Error()
if strings.Contains(msg, "not found") {
return fuse.Status(syscall.ENOENT)
} else if strings.Contains(msg, "not empty") {
return fuse.Status(syscall.ENOTEMPTY)
} else if strings.Contains(msg, "not directory") {
return fuse.ENOTDIR
}
return fuse.EIO
}
wfs.inodeToPath.TouchDirectory(oldDir)
wfs.inodeToPath.TouchDirectory(newDir)

66
weed/mount/weedfs_stream_helpers.go Normal file
View File

@@ -0,0 +1,66 @@
package mount
import (
"context"
"errors"
"github.com/seaweedfs/seaweedfs/weed/glog"
"github.com/seaweedfs/seaweedfs/weed/pb/filer_pb"
)
// streamCreateEntry routes a CreateEntryRequest through the streaming mux
// if available, falling back to a unary gRPC call on transport errors.
func (wfs *WFS) streamCreateEntry(ctx context.Context, req *filer_pb.CreateEntryRequest) (*filer_pb.CreateEntryResponse, error) {
if wfs.streamMutate != nil && wfs.streamMutate.IsAvailable() {
resp, err := wfs.streamMutate.CreateEntry(ctx, req)
if err == nil || !errors.Is(err, ErrStreamTransport) {
return resp, err // success or application error — don't retry
}
glog.V(1).Infof("streamCreateEntry %s/%s: stream failed, falling back to unary: %v", req.Directory, req.Entry.Name, err)
}
var resp *filer_pb.CreateEntryResponse
err := wfs.WithFilerClient(false, func(client filer_pb.SeaweedFilerClient) error {
var err error
resp, err = filer_pb.CreateEntryWithResponse(ctx, client, req)
return err
})
return resp, err
}
// streamUpdateEntry routes an UpdateEntryRequest through the streaming mux
// if available, falling back to a unary gRPC call on transport errors.
func (wfs *WFS) streamUpdateEntry(ctx context.Context, req *filer_pb.UpdateEntryRequest) (*filer_pb.UpdateEntryResponse, error) {
if wfs.streamMutate != nil && wfs.streamMutate.IsAvailable() {
resp, err := wfs.streamMutate.UpdateEntry(ctx, req)
if err == nil || !errors.Is(err, ErrStreamTransport) {
return resp, err
}
glog.V(1).Infof("streamUpdateEntry %s/%s: stream failed, falling back to unary: %v", req.Directory, req.Entry.Name, err)
}
var resp *filer_pb.UpdateEntryResponse
err := wfs.WithFilerClient(false, func(client filer_pb.SeaweedFilerClient) error {
var err error
resp, err = client.UpdateEntry(ctx, req)
return err
})
return resp, err
}
// streamDeleteEntry routes a DeleteEntryRequest through the streaming mux
// if available, falling back to a unary gRPC call on transport errors.
func (wfs *WFS) streamDeleteEntry(ctx context.Context, req *filer_pb.DeleteEntryRequest) (*filer_pb.DeleteEntryResponse, error) {
if wfs.streamMutate != nil && wfs.streamMutate.IsAvailable() {
resp, err := wfs.streamMutate.DeleteEntry(ctx, req)
if err == nil || !errors.Is(err, ErrStreamTransport) {
return resp, err
}
glog.V(1).Infof("streamDeleteEntry %s/%s: stream failed, falling back to unary: %v", req.Directory, req.Name, err)
}
var resp *filer_pb.DeleteEntryResponse
err := wfs.WithFilerClient(false, func(client filer_pb.SeaweedFilerClient) error {
var err error
resp, err = client.DeleteEntry(ctx, req)
return err
})
return resp, err
}

506
weed/mount/weedfs_stream_mutate.go Normal file
View File

@@ -0,0 +1,506 @@
package mount
import (
"context"
"errors"
"fmt"
"io"
"sync"
"sync/atomic"
"syscall"
"github.com/seaweedfs/seaweedfs/weed/glog"
"github.com/seaweedfs/seaweedfs/weed/pb"
"github.com/seaweedfs/seaweedfs/weed/pb/filer_pb"
"google.golang.org/grpc"
"google.golang.org/grpc/codes"
grpcMetadata "google.golang.org/grpc/metadata"
"google.golang.org/grpc/status"
)
// streamMutateError is returned when the server reports a structured errno.
// It is also used by helpers to distinguish application errors (don't retry
// on unary fallback) from transport errors (do retry).
type streamMutateError struct {
msg string
errno syscall.Errno
}
func (e *streamMutateError) Error() string { return e.msg }
func (e *streamMutateError) Errno() syscall.Errno { return e.errno }
// ErrStreamTransport is a sentinel error type for transport-level stream
// failures (disconnects, send errors). Callers use errors.Is to decide
// whether to fall back to unary RPCs.
var ErrStreamTransport = errors.New("stream transport error")
// streamMutateMux multiplexes filer mutation RPCs (create, update, delete,
// rename) over a single bidirectional gRPC stream. Multiple goroutines can
// call the mutation methods concurrently; requests are serialized through
// sendCh and responses are dispatched back via per-request channels.
type streamMutateMux struct {
wfs *WFS
mu sync.Mutex // protects stream, cancel, grpcConn, closed, stopSend, generation
stream filer_pb.SeaweedFiler_StreamMutateEntryClient
cancel context.CancelFunc
grpcConn *grpc.ClientConn // dedicated connection, closed on stream teardown
closed bool
disabled bool // permanently disabled if filer doesn't support the RPC
stopSend chan struct{} // closed to signal the current sendLoop to exit
generation uint64 // incremented each time a new stream is created
nextID atomic.Uint64
// pending maps request_id → response channel. The recvLoop dispatches
// each response to the correct waiter. For rename (multi-response),
// the channel receives multiple messages until is_last=true.
pending sync.Map // map[uint64]chan *filer_pb.StreamMutateEntryResponse
sendCh chan *streamMutateReq
recvDone chan struct{} // closed when recvLoop exits
}
type streamMutateReq struct {
req *filer_pb.StreamMutateEntryRequest
errCh chan error // send error feedback
gen uint64 // stream generation this request targets
}
func newStreamMutateMux(wfs *WFS) *streamMutateMux {
return &streamMutateMux{
wfs: wfs,
sendCh: make(chan *streamMutateReq, 512),
}
}
// CreateEntry sends a CreateEntryRequest over the stream and waits for the response.
func (m *streamMutateMux) CreateEntry(ctx context.Context, req *filer_pb.CreateEntryRequest) (*filer_pb.CreateEntryResponse, error) {
resp, err := m.doUnary(ctx, &filer_pb.StreamMutateEntryRequest{
Request: &filer_pb.StreamMutateEntryRequest_CreateRequest{CreateRequest: req},
})
if err != nil {
return nil, err
}
r, ok := resp.Response.(*filer_pb.StreamMutateEntryResponse_CreateResponse)
if !ok {
return nil, fmt.Errorf("unexpected response type %T", resp.Response)
}
// Check nested error fields (same logic as CreateEntryWithResponse).
cr := r.CreateResponse
if cr.ErrorCode != filer_pb.FilerError_OK {
if sentinel := filer_pb.FilerErrorToSentinel(cr.ErrorCode); sentinel != nil {
return nil, fmt.Errorf("CreateEntry %s/%s: %w", req.Directory, req.Entry.Name, sentinel)
}
return nil, &streamMutateError{msg: cr.Error, errno: syscall.EIO}
}
if cr.Error != "" {
return nil, &streamMutateError{msg: cr.Error, errno: syscall.EIO}
}
return cr, nil
}
// UpdateEntry sends an UpdateEntryRequest over the stream and waits for the response.
func (m *streamMutateMux) UpdateEntry(ctx context.Context, req *filer_pb.UpdateEntryRequest) (*filer_pb.UpdateEntryResponse, error) {
resp, err := m.doUnary(ctx, &filer_pb.StreamMutateEntryRequest{
Request: &filer_pb.StreamMutateEntryRequest_UpdateRequest{UpdateRequest: req},
})
if err != nil {
return nil, err
}
if r, ok := resp.Response.(*filer_pb.StreamMutateEntryResponse_UpdateResponse); ok {
return r.UpdateResponse, nil
}
return nil, fmt.Errorf("unexpected response type %T", resp.Response)
}
// DeleteEntry sends a DeleteEntryRequest over the stream and waits for the response.
func (m *streamMutateMux) DeleteEntry(ctx context.Context, req *filer_pb.DeleteEntryRequest) (*filer_pb.DeleteEntryResponse, error) {
resp, err := m.doUnary(ctx, &filer_pb.StreamMutateEntryRequest{
Request: &filer_pb.StreamMutateEntryRequest_DeleteRequest{DeleteRequest: req},
})
if err != nil {
return nil, err
}
r, ok := resp.Response.(*filer_pb.StreamMutateEntryResponse_DeleteResponse)
if !ok {
return nil, fmt.Errorf("unexpected response type %T", resp.Response)
}
// Check nested error field.
if r.DeleteResponse.Error != "" {
return nil, &streamMutateError{msg: r.DeleteResponse.Error, errno: syscall.EIO}
}
return r.DeleteResponse, nil
}
// Rename sends a StreamRenameEntryRequest over the stream and collects all
// response events until is_last=true. The callback is invoked for each
// intermediate rename event (same as the current StreamRenameEntry recv loop).
func (m *streamMutateMux) Rename(ctx context.Context, req *filer_pb.StreamRenameEntryRequest, onEvent func(*filer_pb.StreamRenameEntryResponse) error) error {
gen, err := m.ensureStream()
if err != nil {
return fmt.Errorf("%w: %v", ErrStreamTransport, err)
}
id := m.nextID.Add(1)
ch := make(chan *filer_pb.StreamMutateEntryResponse, 64)
m.pending.Store(id, ch)
defer m.pending.Delete(id)
sendReq := &streamMutateReq{
req: &filer_pb.StreamMutateEntryRequest{
RequestId: id,
Request: &filer_pb.StreamMutateEntryRequest_RenameRequest{RenameRequest: req},
},
errCh: make(chan error, 1),
gen: gen,
}
select {
case m.sendCh <- sendReq:
case <-ctx.Done():
return ctx.Err()
}
select {
case err := <-sendReq.errCh:
if err != nil {
return fmt.Errorf("rename send: %w: %v", ErrStreamTransport, err)
}
case <-ctx.Done():
return ctx.Err()
}
// Collect rename events until is_last=true.
for {
select {
case resp, ok := <-ch:
if !ok {
return fmt.Errorf("rename recv: %w: stream closed", ErrStreamTransport)
}
if r, ok := resp.Response.(*filer_pb.StreamMutateEntryResponse_RenameResponse); ok {
if r.RenameResponse != nil && r.RenameResponse.EventNotification != nil {
if err := onEvent(r.RenameResponse); err != nil {
return err
}
}
}
if resp.IsLast {
if resp.Error != "" {
return &streamMutateError{
msg: resp.Error,
errno: syscall.Errno(resp.Errno),
}
}
return nil
}
case <-ctx.Done():
return ctx.Err()
}
}
}
// doUnary sends a single-response request and waits for the reply.
func (m *streamMutateMux) doUnary(ctx context.Context, req *filer_pb.StreamMutateEntryRequest) (*filer_pb.StreamMutateEntryResponse, error) {
gen, err := m.ensureStream()
if err != nil {
return nil, fmt.Errorf("%w: %v", ErrStreamTransport, err)
}
id := m.nextID.Add(1)
req.RequestId = id
ch := make(chan *filer_pb.StreamMutateEntryResponse, 1)
m.pending.Store(id, ch)
defer m.pending.Delete(id)
sendReq := &streamMutateReq{
req: req,
errCh: make(chan error, 1),
gen: gen,
}
select {
case m.sendCh <- sendReq:
case <-ctx.Done():
return nil, ctx.Err()
}
select {
case err := <-sendReq.errCh:
if err != nil {
return nil, fmt.Errorf("%w: %v", ErrStreamTransport, err)
}
case <-ctx.Done():
return nil, ctx.Err()
}
select {
case resp, ok := <-ch:
if !ok {
return nil, fmt.Errorf("%w: stream closed", ErrStreamTransport)
}
if resp.Error != "" {
return nil, &streamMutateError{
msg: resp.Error,
errno: syscall.Errno(resp.Errno),
}
}
return resp, nil
case <-ctx.Done():
return nil, ctx.Err()
}
}
// ensureStream opens the bidi stream if not already open. It returns the
// stream generation so callers can tag outgoing requests.
func (m *streamMutateMux) ensureStream() (uint64, error) {
m.mu.Lock()
defer m.mu.Unlock()
if m.closed {
return 0, fmt.Errorf("stream mux is closed")
}
if m.disabled {
return 0, fmt.Errorf("StreamMutateEntry not supported by filer")
}
if m.stream != nil {
return m.generation, nil
}
// Wait for prior generation's recvLoop to fully tear down before opening
// a new stream. This guarantees all pending waiters from the old stream
// have been failed before we create a new generation.
if m.recvDone != nil {
done := m.recvDone
m.mu.Unlock()
<-done
m.mu.Lock()
// Re-check after reacquiring the lock.
if m.closed {
return 0, fmt.Errorf("stream mux is closed")
}
if m.disabled {
return 0, fmt.Errorf("StreamMutateEntry not supported by filer")
}
if m.stream != nil {
return m.generation, nil
}
}
var stream filer_pb.SeaweedFiler_StreamMutateEntryClient
err := m.openStream(&stream)
if err != nil {
if s, ok := status.FromError(err); ok && s.Code() == codes.Unimplemented {
m.disabled = true
glog.V(0).Infof("filer does not support StreamMutateEntry, falling back to unary RPCs")
}
return 0, err
}
m.generation++
m.stream = stream
m.stopSend = make(chan struct{})
recvDone := make(chan struct{})
m.recvDone = recvDone
gen := m.generation
go m.sendLoop(stream, m.stopSend, gen)
go m.recvLoop(stream, gen, recvDone)
return gen, nil
}
func (m *streamMutateMux) openStream(out *filer_pb.SeaweedFiler_StreamMutateEntryClient) error {
i := atomic.LoadInt32(&m.wfs.option.filerIndex)
n := int32(len(m.wfs.option.FilerAddresses))
var lastErr error
for x := int32(0); x < n; x++ {
idx := (i + x) % n
filerGrpcAddress := m.wfs.option.FilerAddresses[idx].ToGrpcAddress()
ctx := context.Background()
if m.wfs.signature != 0 {
ctx = grpcMetadata.AppendToOutgoingContext(ctx, "sw-client-id", fmt.Sprintf("%d", m.wfs.signature))
}
grpcConn, err := pb.GrpcDial(ctx, filerGrpcAddress, false, m.wfs.option.GrpcDialOption)
if err != nil {
lastErr = fmt.Errorf("stream dial %s: %v", filerGrpcAddress, err)
continue
}
client := filer_pb.NewSeaweedFilerClient(grpcConn)
streamCtx, cancel := context.WithCancel(ctx)
stream, err := client.StreamMutateEntry(streamCtx)
if err != nil {
cancel()
grpcConn.Close()
lastErr = err
// Unimplemented means all filers lack it — stop rotating.
if s, ok := status.FromError(err); ok && s.Code() == codes.Unimplemented {
return err
}
continue
}
atomic.StoreInt32(&m.wfs.option.filerIndex, idx)
m.cancel = cancel
m.grpcConn = grpcConn
*out = stream
return nil
}
return lastErr
}
func (m *streamMutateMux) sendLoop(stream filer_pb.SeaweedFiler_StreamMutateEntryClient, stop <-chan struct{}, gen uint64) {
defer m.drainSendCh()
for {
select {
case req, ok := <-m.sendCh:
if !ok {
return // defensive: sendCh should not be closed
}
if req.gen != gen {
req.errCh <- fmt.Errorf("%w: stream generation mismatch", ErrStreamTransport)
continue
}
err := stream.Send(req.req)
req.errCh <- err
if err != nil {
m.teardownStream(gen)
return
}
case <-stop:
return
}
}
}
func (m *streamMutateMux) recvLoop(stream filer_pb.SeaweedFiler_StreamMutateEntryClient, gen uint64, recvDone chan struct{}) {
defer func() {
m.failAllPending()
close(recvDone)
}()
for {
resp, err := stream.Recv()
if err != nil {
if err != io.EOF {
glog.V(1).Infof("stream mutate recv error (gen=%d): %v", gen, err)
}
m.teardownStream(gen)
return
}
if ch, ok := m.pending.Load(resp.RequestId); ok {
ch.(chan *filer_pb.StreamMutateEntryResponse) <- resp
// For single-response ops, the caller deletes from pending after recv.
// For rename, the caller collects until is_last.
}
}
}
// teardownStream cleans up the stream for the given generation. It is safe to
// call from both sendLoop and recvLoop; only the first call for a given
// generation takes effect (idempotent via generation + nil-stream check).
func (m *streamMutateMux) teardownStream(gen uint64) {
m.mu.Lock()
if m.generation != gen || m.stream == nil {
m.mu.Unlock()
return
}
m.stream = nil
if m.stopSend != nil {
close(m.stopSend)
m.stopSend = nil
}
if m.cancel != nil {
m.cancel()
m.cancel = nil
}
conn := m.grpcConn
m.grpcConn = nil
m.mu.Unlock()
// Do NOT call failAllPending here — recvLoop is the sole owner of
// pending channel teardown. This avoids a race where teardownStream
// closes a channel that recvLoop is about to send on.
if conn != nil {
conn.Close()
}
}
// failAllPending closes all pending response channels, causing waiters to
// receive ok=false. It is idempotent: entries are deleted before channels are
// closed, so concurrent calls cannot double-close.
func (m *streamMutateMux) failAllPending() {
var channels []chan *filer_pb.StreamMutateEntryResponse
m.pending.Range(func(key, value any) bool {
m.pending.Delete(key)
channels = append(channels, value.(chan *filer_pb.StreamMutateEntryResponse))
return true
})
for _, ch := range channels {
close(ch)
}
}
// drainSendCh drains buffered requests from sendCh, sending an error to each
// request's errCh so callers don't block. Called by sendLoop's defer on exit
// and by Close for any stragglers.
func (m *streamMutateMux) drainSendCh() {
for {
select {
case req, ok := <-m.sendCh:
if !ok {
return // defensive: sendCh should not be closed
}
req.errCh <- fmt.Errorf("%w: stream shutting down", ErrStreamTransport)
default:
return
}
}
}
// IsAvailable returns true if the stream mux is usable (not permanently disabled).
func (m *streamMutateMux) IsAvailable() bool {
m.mu.Lock()
defer m.mu.Unlock()
return !m.disabled
}
// Close shuts down the stream. Called during unmount after all flushes complete.
func (m *streamMutateMux) Close() {
m.mu.Lock()
if m.closed {
m.mu.Unlock()
return
}
m.closed = true
stream := m.stream
m.stream = nil // prevent teardownStream from acting after Close
cancel := m.cancel
m.cancel = nil
grpcConn := m.grpcConn
m.grpcConn = nil
recvDone := m.recvDone
if m.stopSend != nil {
close(m.stopSend)
m.stopSend = nil
}
m.mu.Unlock()
// CloseSend triggers EOF on recvLoop; cancel ensures Recv unblocks
// even if the transport is broken.
if stream != nil {
stream.CloseSend()
}
if cancel != nil {
cancel()
}
// Wait for recvLoop to finish — it calls failAllPending on exit.
if recvDone != nil {
<-recvDone
}
if grpcConn != nil {
grpcConn.Close()
}
// Drain any remaining requests buffered in sendCh. sendLoop's defer
// drain handles most items, but stragglers enqueued during shutdown
// (between ensureStream and the sendCh send) are caught here.
// sendCh is intentionally left open to prevent send-on-closed panics.
m.drainSendCh()
}

19
weed/mount/weedfs_symlink.go
View File

@@ -2,7 +2,6 @@ package mount
import (
"context"
"fmt"
"os"
"syscall"
"time"
@@ -47,16 +46,10 @@ func (wfs *WFS) Symlink(cancel <-chan struct{}, header *fuse.InHeader, target st
SkipCheckParentDirectory: true,
}
err := wfs.WithFilerClient(false, func(client filer_pb.SeaweedFilerClient) error {
wfs.mapPbIdFromLocalToFiler(request.Entry)
defer wfs.mapPbIdFromFilerToLocal(request.Entry)
resp, err := filer_pb.CreateEntryWithResponse(context.Background(), client, request)
if err != nil {
return fmt.Errorf("symlink %s: %v", entryFullPath, err)
}
wfs.mapPbIdFromLocalToFiler(request.Entry)
resp, err := wfs.streamCreateEntry(context.Background(), request)
if err == nil {
event := resp.GetMetadataEvent()
if event == nil {
event = metadataCreateEvent(string(dirPath), request.Entry)
@@ -65,9 +58,11 @@ func (wfs *WFS) Symlink(cancel <-chan struct{}, header *fuse.InHeader, target st
glog.Warningf("symlink %s: best-effort metadata apply failed: %v", entryFullPath, applyErr)
wfs.inodeToPath.InvalidateChildrenCache(dirPath)
}
}
// Map back to local uid/gid before writing to the kernel.
wfs.mapPbIdFromFilerToLocal(request.Entry)
return nil
})
if err != nil {
glog.V(0).Infof("Symlink %s => %s: %v", entryFullPath, target, err)
return fuse.EIO

31
weed/mount/wfs_save.go
View File

@@ -15,23 +15,20 @@ func (wfs *WFS) saveEntry(path util.FullPath, entry *filer_pb.Entry) (code fuse.
parentDir, _ := path.DirAndName()
err := wfs.WithFilerClient(false, func(client filer_pb.SeaweedFilerClient) error {
wfs.mapPbIdFromLocalToFiler(entry)
defer wfs.mapPbIdFromFilerToLocal(entry)
wfs.mapPbIdFromLocalToFiler(entry)
defer wfs.mapPbIdFromFilerToLocal(entry)
request := &filer_pb.UpdateEntryRequest{
Directory: parentDir,
Entry: entry,
Signatures: []int32{wfs.signature},
}
glog.V(1).Infof("save entry: %v", request)
resp, err := filer_pb.UpdateEntryWithResponse(context.Background(), client, request)
if err != nil {
return fmt.Errorf("UpdateEntry dir %s: %v", path, err)
}
request := &filer_pb.UpdateEntryRequest{
Directory: parentDir,
Entry: entry,
Signatures: []int32{wfs.signature},
}
glog.V(1).Infof("save entry: %v", request)
resp, err := wfs.streamUpdateEntry(context.Background(), request)
if err != nil {
err = fmt.Errorf("UpdateEntry dir %s: %v", path, err)
} else {
event := resp.GetMetadataEvent()
if event == nil {
event = metadataUpdateEvent(parentDir, entry)
@@ -40,9 +37,7 @@ func (wfs *WFS) saveEntry(path util.FullPath, entry *filer_pb.Entry) (code fuse.
glog.Warningf("saveEntry %s: best-effort metadata apply failed: %v", path, applyErr)
wfs.inodeToPath.InvalidateChildrenCache(util.FullPath(parentDir))
}
return nil
})
}
if err != nil {
// glog.V(0).Infof("saveEntry %s: %v", path, err)
fuseStatus := grpcErrorToFuseStatus(err)