* Add Trino blog operations test * Update test/s3tables/catalog_trino/trino_blog_operations_test.go Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * feat: add table bucket path helpers and filer operations - Add table object root and table location mapping directories - Implement ensureDirectory, upsertFile, deleteEntryIfExists helpers - Support table location bucket mapping for S3 access * feat: manage table bucket object roots on creation/deletion - Create .objects directory for table buckets on creation - Clean up table object bucket paths on deletion - Enable S3 operations on table bucket object roots * feat: add table location mapping for Iceberg REST - Track table location bucket mappings when tables are created/updated/deleted - Enable location-based routing for S3 operations on table data * feat: route S3 operations to table bucket object roots - Route table-s3 bucket names to mapped table paths - Route table buckets to object root directories - Support table location bucket mapping lookup * feat: emit table-s3 locations from Iceberg REST - Generate unique table-s3 bucket names with UUID suffix - Store table metadata under table bucket paths - Return table-s3 locations for Trino compatibility * fix: handle missing directories in S3 list operations - Propagate ErrNotFound from ListEntries for non-existent directories - Treat missing directories as empty results for list operations - Fixes Trino non-empty location checks on table creation * test: improve Trino CSV parsing for single-value results - Sanitize Trino output to skip jline warnings - Handle single-value CSV results without header rows - Strip quotes from numeric values in tests * refactor: use bucket path helpers throughout S3 API - Replace direct bucket path operations with helper functions - Leverage centralized table bucket routing logic - Improve maintainability with consistent path resolution * fix: add table bucket cache and improve filer error handling - Cache table bucket lookups to reduce filer overhead on repeated checks - Use filer_pb.CreateEntry and filer_pb.UpdateEntry helpers to check resp.Error - Fix delete order in handler_bucket_get_list_delete: delete table object before directory - Make location mapping errors best-effort: log and continue, don't fail API - Update table location mappings to delete stale prior bucket mappings on update - Add 1-second sleep before timestamp time travel query to ensure timestamps are in past - Fix CSV parsing: examine all lines, not skip first; handle single-value rows * fix: properly handle stale metadata location mapping cleanup - Capture oldMetadataLocation before mutation in handleUpdateTable - Update updateTableLocationMapping to accept both old and new locations - Use passed-in oldMetadataLocation to detect location changes - Delete stale mapping only when location actually changes - Pass empty string for oldLocation in handleCreateTable (new tables have no prior mapping) - Improve logging to show old -> new location transitions * refactor: cleanup imports and cache design - Remove unused 'sync' import from bucket_paths.go - Use filer_pb.UpdateEntry helper in setExtendedAttribute and deleteExtendedAttribute for consistent error handling - Add dedicated tableBucketCache map[string]bool to BucketRegistry instead of mixing concerns with metadataCache - Improve cache separation: table buckets cache is now separate from bucket metadata cache * fix: improve cache invalidation and add transient error handling Cache invalidation (critical fix): - Add tableLocationCache to BucketRegistry for location mapping lookups - Clear tableBucketCache and tableLocationCache in RemoveBucketMetadata - Prevents stale cache entries when buckets are deleted/recreated Transient error handling: - Only cache table bucket lookups when conclusive (found or ErrNotFound) - Skip caching on transient errors (network, permission, etc) - Prevents marking real table buckets as non-table due to transient failures Performance optimization: - Cache tableLocationDir results to avoid repeated filer RPCs on hot paths - tableLocationDir now checks cache before making expensive filer lookups - Cache stores empty string for 'not found' to avoid redundant lookups Code clarity: - Add comment to deleteDirectory explaining DeleteEntry response lacks Error field * go fmt * fix: mirror transient error handling in tableLocationDir and optimize bucketDir Transient error handling: - tableLocationDir now only caches definitive results - Mirrors isTableBucket behavior to prevent treating transient errors as permanent misses - Improves reliability on flaky systems or during recovery Performance optimization: - bucketDir avoids redundant isTableBucket call via bucketRoot - Directly use s3a.option.BucketsPath for regular buckets - Saves one cache lookup for every non-table bucket operation * fix: revert bucketDir optimization to preserve bucketRoot logic The optimization to directly use BucketsPath bypassed bucketRoot's logic and caused issues with S3 list operations on delimiter+prefix cases. Revert to using path.Join(s3a.bucketRoot(bucket), bucket) which properly handles all bucket types and ensures consistent path resolution across the codebase. The slight performance cost of an extra cache lookup is worth the correctness and consistency benefits. * feat: move table buckets under /buckets Add a table-bucket marker attribute, reuse bucket metadata cache for table bucket detection, and update list/validation/UI/test paths to treat table buckets as /buckets entries. * Fix S3 Tables code review issues - handler_bucket_create.go: Fix bucket existence check to properly validate entryResp.Entry before setting s3BucketExists flag (nil Entry should not indicate existing bucket) - bucket_paths.go: Add clarifying comment to bucketRoot() explaining unified buckets root path for all bucket types - file_browser_data.go: Optimize by extracting table bucket check early to avoid redundant WithFilerClient call * Fix list prefix delimiter handling * Handle list errors conservatively * Fix Trino FOR TIMESTAMP query - use past timestamp Iceberg requires the timestamp to be strictly in the past. Use current_timestamp - interval '1' second instead of current_timestamp. --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
SeaweedFS Admin Component
A modern web-based administration interface for SeaweedFS clusters built with Go, Gin, Templ, and Bootstrap.
Features
- Dashboard: Real-time cluster status and metrics
- Master Management: Monitor master nodes and leadership status
- Volume Server Management: View volume servers, capacity, and health
- Object Store Bucket Management: Create, delete, and manage Object Store buckets with web interface
- S3 Tables Management: Manage table buckets, namespaces, tables, tags, and policies via the admin UI
- System Health: Overall cluster health monitoring
- Responsive Design: Bootstrap-based UI that works on all devices
- Authentication: Optional user authentication with sessions
- TLS Support: HTTPS support for production deployments
Building
Using the Admin Makefile
The admin component has its own Makefile for development and building:
# Navigate to admin directory
cd weed/admin
# View all available targets
make help
# Generate templates and build
make build
# Development mode with template watching
make dev
# Run the admin server
make run
# Clean build artifacts
make clean
Using the Root Makefile
The root SeaweedFS Makefile automatically integrates the admin component:
# From the root directory
make install # Builds weed with admin component
make full_install # Full build with all tags
make test # Runs tests including admin component
# Admin-specific targets from root
make admin-generate # Generate admin templates
make admin-build # Build admin component
make admin-run # Run admin server
make admin-dev # Development mode
make admin-clean # Clean admin artifacts
Manual Building
If you prefer to build manually:
# Install templ compiler
go install github.com/a-h/templ/cmd/templ@latest
# Generate templates
templ generate
# Build the main weed binary
cd ../../../
go build -o weed ./weed
Development
Template Development
The admin interface uses Templ for type-safe HTML templates:
# Watch for template changes and auto-regenerate
make watch
# Or manually generate templates
make generate
# Format templates
make fmt
File Structure
weed/admin/
├── Makefile # Admin-specific build tasks
├── README.md # This file
├── admin.go # Main application entry point
├── dash/ # Server and handler logic
│ ├── admin_server.go # HTTP server setup
│ ├── handler_admin.go # Admin dashboard handlers
│ ├── handler_auth.go # Authentication handlers
│ └── middleware.go # HTTP middleware
├── static/ # Static assets
│ ├── css/admin.css # Admin-specific styles
│ └── js/admin.js # Admin-specific JavaScript
└── view/ # Templates
├── app/ # Application templates
│ ├── admin.templ # Main dashboard template
│ ├── s3_buckets.templ # Object Store bucket management template
│ ├── s3tables_*.templ # S3 Tables management templates
│ └── *_templ.go # Generated Go code
└── layout/ # Layout templates
├── layout.templ # Base layout template
└── layout_templ.go # Generated Go code
Object Store Management
The admin interface includes Object Store and S3 Tables management capabilities:
- Create/delete Object Store buckets and adjust quotas or ownership.
- Manage S3 Tables buckets, namespaces, and tables.
- Update S3 Tables policies and tags via the UI and API endpoints.
Usage
Basic Usage
# Start admin interface on default port (23646)
weed admin
# Start with custom configuration
weed admin -port=8080 -masters="master1:9333,master2:9333"
# Start with authentication
weed admin -adminUser=admin -adminPassword=secret123
# Start with HTTPS
weed admin -port=443 -tlsCert=/path/to/cert.pem -tlsKey=/path/to/key.pem
Configuration Options
| Option | Default | Description |
|---|---|---|
-port |
23646 | Admin server port |
-masters |
localhost:9333 | Comma-separated master servers |
-adminUser |
admin | Admin username (if auth enabled) |
-adminPassword |
"" | Admin password (empty = no auth) |
-tlsCert |
"" | Path to TLS certificate |
-tlsKey |
"" | Path to TLS private key |
Docker Usage
# Build Docker image with admin component
make docker-build
# Run with Docker
docker run -p 23646:23646 seaweedfs/seaweedfs:latest admin -masters=host.docker.internal:9333
Development Workflow
Quick Start
# Clone and setup
git clone <seaweedfs-repo>
cd seaweedfs/weed/admin
# Install dependencies and build
make install-deps
make build
# Start development server
make dev
Making Changes
-
Template Changes: Edit
.templfiles inview/- Templates auto-regenerate in development mode
- Use
make generateto manually regenerate
-
Go Code Changes: Edit
.gofiles- Restart the server to see changes
- Use
make buildto rebuild
-
Static Assets: Edit files in
static/- Changes are served immediately
Testing
# Run admin component tests
make test
# Run from root directory
make admin-test
# Lint code
make lint
# Format code
make fmt
Production Deployment
Security Considerations
- Authentication: Always set
adminPasswordin production - HTTPS: Use TLS certificates for encrypted connections
- Firewall: Restrict admin interface access to authorized networks
Example Production Setup
# Production deployment with security
weed admin \
-port=443 \
-masters="master1:9333,master2:9333,master3:9333" \
-adminUser=admin \
-adminPassword=your-secure-password \
-tlsCert=/etc/ssl/certs/admin.crt \
-tlsKey=/etc/ssl/private/admin.key
Monitoring
The admin interface provides endpoints for monitoring:
GET /health- Health check endpointGET /metrics- Prometheus metrics (if enabled)GET /api/status- JSON status information
Troubleshooting
Common Issues
- Templates not found: Run
make generateto create template files - Build errors: Ensure
templis installed withmake install-templ - Static files not loading: Check that
static/directory exists and has proper files - Connection errors: Verify master and filer addresses are correct
Debug Mode
# Enable debug logging
weed -v=2 admin
# Check generated templates
ls -la view/app/*_templ.go view/layout/*_templ.go
Contributing
- Fork the repository
- Create a feature branch
- Make your changes
- Run tests:
make test - Format code:
make fmt - Submit a pull request
Architecture
The admin component follows a clean architecture:
- Presentation Layer: Templ templates + Bootstrap CSS
- HTTP Layer: Gin router with middleware
- Business Logic: Handler functions in
dash/package - Data Layer: Communicates with SeaweedFS masters and filers
This separation makes the code maintainable and testable.