fix: S3 listing NextMarker missing intermediate directory component (#8089)

* fix: S3 listing NextMarker missing intermediate directory component

When listing with nested prefixes like "character/member/", the NextMarker
was incorrectly constructed as "character/res024/" instead of
"character/member/res024/", causing continuation requests to fail.

Root cause: The code at line 331 was constructing NextMarker as:
  nextMarker = requestDir + "/" + nextMarker

This worked when nextMarker already contained the full relative path,
but failed when it was just the entry name from the innermost recursion.

Fix: Include the prefix component when constructing NextMarker:
  if prefix != "" {
      nextMarker = requestDir + "/" + prefix + "/" + nextMarker
  }

This ensures the full path is always constructed correctly for both:
- CommonPrefix entries (directories)
- Regular entries (files)

Also includes fix for cursor.prefixEndsOnDelimiter state leak that was
causing sibling directories to be incorrectly listed.

* test: add regression tests for NextMarker construction

Add comprehensive unit tests to verify NextMarker is correctly constructed
with nested prefixes. Tests cover:
- Regular entries with nested prefix (character/member/res024)
- CommonPrefix entries (directories)
- Edge cases (no requestDir, no prefix, deeply nested)

These tests ensure the fix prevents regression of the bug where
NextMarker was missing intermediate directory components.
This commit is contained in:
Chris Lu
2026-01-22 16:56:35 -08:00
committed by GitHub
parent 066410dbd0
commit bc1113208d
2 changed files with 130 additions and 2 deletions

View File

@@ -322,13 +322,21 @@ func (s3a *S3ApiServer) listFilerEntries(bucket string, originalPrefix string, m
if cursor.isTruncated && lastEntryWasCommonPrefix && lastCommonPrefixName != "" {
// For CommonPrefixes, NextMarker should include the trailing slash
if requestDir != "" {
nextMarker = requestDir + "/" + lastCommonPrefixName + "/"
if prefix != "" {
nextMarker = requestDir + "/" + prefix + "/" + lastCommonPrefixName + "/"
} else {
nextMarker = requestDir + "/" + lastCommonPrefixName + "/"
}
} else {
nextMarker = lastCommonPrefixName + "/"
}
} else if cursor.isTruncated {
if requestDir != "" {
nextMarker = requestDir + "/" + nextMarker
if prefix != "" {
nextMarker = requestDir + "/" + prefix + "/" + nextMarker
} else {
nextMarker = requestDir + "/" + nextMarker
}
}
}