2bb21ea276
* feat: Add Iceberg REST Catalog server Implement Iceberg REST Catalog API on a separate port (default 8181) that exposes S3 Tables metadata through the Apache Iceberg REST protocol. - Add new weed/s3api/iceberg package with REST handlers - Implement /v1/config endpoint returning catalog configuration - Implement namespace endpoints (list/create/get/head/delete) - Implement table endpoints (list/create/load/head/delete/update) - Add -port.iceberg flag to S3 standalone server (s3.go) - Add -s3.port.iceberg flag to combined server mode (server.go) - Add -s3.port.iceberg flag to mini cluster mode (mini.go) - Support prefix-based routing for multiple catalogs The Iceberg REST server reuses S3 Tables metadata storage under /table-buckets and enables DuckDB, Spark, and other Iceberg clients to connect to SeaweedFS as a catalog. * feat: Add Iceberg Catalog pages to admin UI Add admin UI pages to browse Iceberg catalogs, namespaces, and tables. - Add Iceberg Catalog menu item under Object Store navigation - Create iceberg_catalog.templ showing catalog overview with REST info - Create iceberg_namespaces.templ listing namespaces in a catalog - Create iceberg_tables.templ listing tables in a namespace - Add handlers and routes in admin_handlers.go - Add Iceberg data provider methods in s3tables_management.go - Add Iceberg data types in types.go The Iceberg Catalog pages provide visibility into the same S3 Tables data through an Iceberg-centric lens, including REST endpoint examples for DuckDB and PyIceberg. * test: Add Iceberg catalog integration tests and reorg s3tables tests - Reorganize existing s3tables tests to test/s3tables/table-buckets/ - Add new test/s3tables/catalog/ for Iceberg REST catalog tests - Add TestIcebergConfig to verify /v1/config endpoint - Add TestIcebergNamespaces to verify namespace listing - Add TestDuckDBIntegration for DuckDB connectivity (requires Docker) - Update CI workflow to use new test paths * fix: Generate proper random UUIDs for Iceberg tables Address code review feedback: - Replace placeholder UUID with crypto/rand-based UUID v4 generation - Add detailed TODO comments for handleUpdateTable stub explaining the required atomic metadata swap implementation * fix: Serve Iceberg on localhost listener when binding to different interface Address code review feedback: properly serve the localhost listener when the Iceberg server is bound to a non-localhost interface. * ci: Add Iceberg catalog integration tests to CI Add new job to run Iceberg catalog tests in CI, along with: - Iceberg package build verification - Iceberg unit tests - Iceberg go vet checks - Iceberg format checks * fix: Address code review feedback for Iceberg implementation - fix: Replace hardcoded account ID with s3_constants.AccountAdminId in buildTableBucketARN() - fix: Improve UUID generation error handling with deterministic fallback (timestamp + PID + counter) - fix: Update handleUpdateTable to return HTTP 501 Not Implemented instead of fake success - fix: Better error handling in handleNamespaceExists to distinguish 404 from 500 errors - fix: Use relative URL in template instead of hardcoded localhost:8181 - fix: Add HTTP timeout to test's waitForService function to avoid hangs - fix: Use dynamic ephemeral ports in integration tests to avoid flaky parallel failures - fix: Add Iceberg port to final port configuration logging in mini.go * fix: Address critical issues in Iceberg implementation - fix: Cache table UUIDs to ensure persistence across LoadTable calls The UUID now remains stable for the lifetime of the server session. TODO: For production, UUIDs should be persisted in S3 Tables metadata. - fix: Remove redundant URL-encoded namespace parsing mux router already decodes %1F to \x1F before passing to handlers. Redundant ReplaceAll call could cause bugs with literal %1F in namespace. * fix: Improve test robustness and reduce code duplication - fix: Make DuckDB test more robust by failing on unexpected errors Instead of silently logging errors, now explicitly check for expected conditions (extension not available) and skip the test appropriately. - fix: Extract username helper method to reduce duplication Created getUsername() helper in AdminHandlers to avoid duplicating the username retrieval logic across Iceberg page handlers. * fix: Add mutex protection to table UUID cache Protects concurrent access to the tableUUIDs map with sync.RWMutex. Uses read-lock for fast path when UUID already cached, and write-lock for generating new UUIDs. Includes double-check pattern to handle race condition between read-unlock and write-lock. * style: fix go fmt errors * feat(iceberg): persist table UUID in S3 Tables metadata * feat(admin): configure Iceberg port in Admin UI and commands * refactor: address review comments (flags, tests, handlers) - command/mini: fix tracking of explicit s3.port.iceberg flag - command/admin: add explicit -iceberg.port flag - admin/handlers: reuse getUsername helper - tests: use 127.0.0.1 for ephemeral ports and os.Stat for file size check * test: check error from FileStat in verify_gc_empty_test
445 lines
23 KiB
Plaintext
445 lines
23 KiB
Plaintext
package layout
|
|
|
|
import (
|
|
"fmt"
|
|
"strings"
|
|
"time"
|
|
"github.com/gin-gonic/gin"
|
|
"github.com/seaweedfs/seaweedfs/weed/util/version"
|
|
)
|
|
|
|
templ Layout(c *gin.Context, content templ.Component) {
|
|
{{
|
|
username := c.GetString("username")
|
|
if username == "" {
|
|
username = "admin"
|
|
}
|
|
|
|
// Detect if we're on a configuration page to keep submenu expanded
|
|
currentPath := c.Request.URL.Path
|
|
isConfigPage := strings.HasPrefix(currentPath, "/maintenance/config") || currentPath == "/config"
|
|
|
|
// Detect if we're on a message queue page to keep submenu expanded
|
|
isMQPage := strings.HasPrefix(currentPath, "/mq/")
|
|
|
|
// Detect if we're on a storage page to keep submenu expanded
|
|
isStoragePage := strings.HasPrefix(currentPath, "/storage/volumes") || strings.HasPrefix(currentPath, "/storage/ec-shards") || strings.HasPrefix(currentPath, "/storage/collections")
|
|
|
|
// Detect if we're on a cluster page (but not storage page) to keep submenu expanded
|
|
isClusterPage := (strings.HasPrefix(currentPath, "/cluster/masters") || strings.HasPrefix(currentPath, "/cluster/volume-servers") || strings.HasPrefix(currentPath, "/cluster/filers"))
|
|
}}
|
|
<!DOCTYPE html>
|
|
<html lang="en">
|
|
<head>
|
|
<meta charset="UTF-8">
|
|
<title>SeaweedFS Admin</title>
|
|
<meta name="viewport" content="width=device-width, initial-scale=1">
|
|
<link rel="icon" href="/static/favicon.ico" type="image/x-icon">
|
|
|
|
<!-- Bootstrap CSS -->
|
|
<link href="/static/css/bootstrap.min.css" rel="stylesheet">
|
|
<!-- Font Awesome CSS -->
|
|
<link href="/static/css/fontawesome.min.css" rel="stylesheet">
|
|
<!-- HTMX -->
|
|
<script src="/static/js/htmx.min.js"></script>
|
|
<!-- Custom CSS -->
|
|
<link rel="stylesheet" href="/static/css/admin.css">
|
|
</head>
|
|
<body>
|
|
<div class="container-fluid">
|
|
<!-- Header -->
|
|
<header class="navbar navbar-expand-lg navbar-dark bg-primary sticky-top">
|
|
<div class="container-fluid">
|
|
<a class="navbar-brand fw-bold" href="/admin">
|
|
<i class="fas fa-server me-2"></i>
|
|
SeaweedFS Admin
|
|
<span class="badge bg-warning text-dark ms-2">ALPHA</span>
|
|
</a>
|
|
|
|
<button class="navbar-toggler" type="button" data-bs-toggle="collapse" data-bs-target="#navbarNav">
|
|
<span class="navbar-toggler-icon"></span>
|
|
</button>
|
|
|
|
<div class="collapse navbar-collapse" id="navbarNav">
|
|
<ul class="navbar-nav ms-auto">
|
|
<li class="nav-item dropdown">
|
|
<a class="nav-link dropdown-toggle" href="#" role="button" data-bs-toggle="dropdown">
|
|
<i class="fas fa-user me-1"></i>{username}
|
|
</a>
|
|
<ul class="dropdown-menu">
|
|
<li><a class="dropdown-item" href="/logout">
|
|
<i class="fas fa-sign-out-alt me-2"></i>Logout
|
|
</a></li>
|
|
</ul>
|
|
</li>
|
|
</ul>
|
|
</div>
|
|
</div>
|
|
</header>
|
|
|
|
<div class="row g-0">
|
|
<!-- Sidebar -->
|
|
<div class="col-md-3 col-lg-2 d-md-block bg-light sidebar collapse">
|
|
<div class="position-sticky pt-3">
|
|
<h6 class="sidebar-heading px-3 mt-4 mb-1 text-muted">
|
|
<span>MAIN</span>
|
|
</h6>
|
|
<ul class="nav flex-column">
|
|
<li class="nav-item">
|
|
<a class="nav-link" href="/admin">
|
|
<i class="fas fa-tachometer-alt me-2"></i>Dashboard
|
|
</a>
|
|
</li>
|
|
<li class="nav-item">
|
|
<a class={ "nav-link", templ.KV("collapsed", !isClusterPage) } href="#" data-bs-toggle="collapse" data-bs-target="#clusterSubmenu" aria-expanded={ fmt.Sprintf("%t", isClusterPage) } aria-controls="clusterSubmenu">
|
|
<i class="fas fa-sitemap me-2"></i>Cluster
|
|
<i class="fas fa-chevron-down ms-auto"></i>
|
|
</a>
|
|
<div class={ "collapse", templ.KV("show", isClusterPage) } id="clusterSubmenu">
|
|
<ul class="nav flex-column ms-3">
|
|
<li class="nav-item">
|
|
<a class="nav-link py-2" href="/cluster/masters">
|
|
<i class="fas fa-crown me-2"></i>Masters
|
|
</a>
|
|
</li>
|
|
<li class="nav-item">
|
|
<a class="nav-link py-2" href="/cluster/volume-servers">
|
|
<i class="fas fa-server me-2"></i>Volume Servers
|
|
</a>
|
|
</li>
|
|
<li class="nav-item">
|
|
<a class="nav-link py-2" href="/cluster/filers">
|
|
<i class="fas fa-folder-open me-2"></i>Filers
|
|
</a>
|
|
</li>
|
|
</ul>
|
|
</div>
|
|
</li>
|
|
<li class="nav-item">
|
|
<a class={ "nav-link", templ.KV("collapsed", !isStoragePage) } href="#" data-bs-toggle="collapse" data-bs-target="#storageSubmenu" aria-expanded={ fmt.Sprintf("%t", isStoragePage) } aria-controls="storageSubmenu">
|
|
<i class="fas fa-hdd me-2"></i>Storage
|
|
<i class="fas fa-chevron-down ms-auto"></i>
|
|
</a>
|
|
<div class={ "collapse", templ.KV("show", isStoragePage) } id="storageSubmenu">
|
|
<ul class="nav flex-column ms-3">
|
|
<li class="nav-item">
|
|
<a class="nav-link py-2" href="/storage/volumes">
|
|
<i class="fas fa-database me-2"></i>Volumes
|
|
</a>
|
|
</li>
|
|
<li class="nav-item">
|
|
<a class="nav-link py-2" href="/storage/ec-shards">
|
|
<i class="fas fa-th-large me-2"></i>EC Volumes
|
|
</a>
|
|
</li>
|
|
<li class="nav-item">
|
|
<a class="nav-link py-2" href="/storage/collections">
|
|
<i class="fas fa-layer-group me-2"></i>Collections
|
|
</a>
|
|
</li>
|
|
</ul>
|
|
</div>
|
|
</li>
|
|
</ul>
|
|
|
|
<h6 class="sidebar-heading px-3 mt-4 mb-1 text-muted">
|
|
<span>MANAGEMENT</span>
|
|
</h6>
|
|
<ul class="nav flex-column">
|
|
<li class="nav-item">
|
|
<a class="nav-link" href="/files">
|
|
<i class="fas fa-folder me-2"></i>File Browser
|
|
</a>
|
|
</li>
|
|
<li class="nav-item">
|
|
<a class="nav-link collapsed" href="#" data-bs-toggle="collapse" data-bs-target="#objectStoreSubmenu" aria-expanded="false" aria-controls="objectStoreSubmenu">
|
|
<i class="fas fa-cloud me-2"></i>Object Store
|
|
<i class="fas fa-chevron-down ms-auto"></i>
|
|
</a>
|
|
<div class="collapse" id="objectStoreSubmenu">
|
|
<ul class="nav flex-column ms-3">
|
|
<li class="nav-item">
|
|
<a class="nav-link py-2" href="/object-store/buckets">
|
|
<i class="fas fa-cube me-2"></i>Buckets
|
|
</a>
|
|
</li>
|
|
<li class="nav-item">
|
|
<a class="nav-link py-2" href="/object-store/s3tables/buckets">
|
|
<i class="fas fa-table me-2"></i>Table Buckets
|
|
</a>
|
|
</li>
|
|
<li class="nav-item">
|
|
<a class="nav-link py-2" href="/object-store/iceberg">
|
|
<i class="fas fa-snowflake me-2"></i>Iceberg Catalog
|
|
</a>
|
|
</li>
|
|
<li class="nav-item">
|
|
<a class="nav-link py-2" href="/object-store/users">
|
|
<i class="fas fa-users me-2"></i>Users
|
|
</a>
|
|
</li>
|
|
<li class="nav-item">
|
|
<a class="nav-link py-2" href="/object-store/service-accounts">
|
|
<i class="fas fa-robot me-2"></i>Service Accounts
|
|
</a>
|
|
</li>
|
|
<li class="nav-item">
|
|
<a class="nav-link py-2" href="/object-store/policies">
|
|
<i class="fas fa-shield-alt me-2"></i>Policies
|
|
</a>
|
|
</li>
|
|
</ul>
|
|
</div>
|
|
</li>
|
|
<li class="nav-item">
|
|
if isMQPage {
|
|
<a class="nav-link" href="#" data-bs-toggle="collapse" data-bs-target="#messageQueueSubmenu" aria-expanded="true" aria-controls="messageQueueSubmenu">
|
|
<i class="fas fa-comments me-2"></i>Message Queue
|
|
<i class="fas fa-chevron-down ms-auto"></i>
|
|
</a>
|
|
} else {
|
|
<a class="nav-link collapsed" href="#" data-bs-toggle="collapse" data-bs-target="#messageQueueSubmenu" aria-expanded="false" aria-controls="messageQueueSubmenu">
|
|
<i class="fas fa-comments me-2"></i>Message Queue
|
|
<i class="fas fa-chevron-down ms-auto"></i>
|
|
</a>
|
|
}
|
|
if isMQPage {
|
|
<div class="collapse show" id="messageQueueSubmenu">
|
|
<ul class="nav flex-column ms-3">
|
|
<li class="nav-item">
|
|
if currentPath == "/mq/brokers" {
|
|
<a class="nav-link py-2 active" href="/mq/brokers">
|
|
<i class="fas fa-server me-2"></i>Brokers
|
|
</a>
|
|
} else {
|
|
<a class="nav-link py-2" href="/mq/brokers">
|
|
<i class="fas fa-server me-2"></i>Brokers
|
|
</a>
|
|
}
|
|
</li>
|
|
<li class="nav-item">
|
|
if currentPath == "/mq/topics" {
|
|
<a class="nav-link py-2 active" href="/mq/topics">
|
|
<i class="fas fa-list-alt me-2"></i>Topics
|
|
</a>
|
|
} else {
|
|
<a class="nav-link py-2" href="/mq/topics">
|
|
<i class="fas fa-list-alt me-2"></i>Topics
|
|
</a>
|
|
}
|
|
</li>
|
|
|
|
</ul>
|
|
</div>
|
|
} else {
|
|
<div class="collapse" id="messageQueueSubmenu">
|
|
<ul class="nav flex-column ms-3">
|
|
<li class="nav-item">
|
|
<a class="nav-link py-2" href="/mq/brokers">
|
|
<i class="fas fa-server me-2"></i>Brokers
|
|
</a>
|
|
</li>
|
|
<li class="nav-item">
|
|
<a class="nav-link py-2" href="/mq/topics">
|
|
<i class="fas fa-list-alt me-2"></i>Topics
|
|
</a>
|
|
</li>
|
|
|
|
</ul>
|
|
</div>
|
|
}
|
|
</li>
|
|
<!-- Commented out for later -->
|
|
<!--
|
|
<li class="nav-item">
|
|
<a class="nav-link" href="/metrics">
|
|
<i class="fas fa-chart-line me-2"></i>Metrics
|
|
</a>
|
|
</li>
|
|
<li class="nav-item">
|
|
<a class="nav-link" href="/logs">
|
|
<i class="fas fa-file-alt me-2"></i>Logs
|
|
</a>
|
|
</li>
|
|
-->
|
|
</ul>
|
|
|
|
<h6 class="sidebar-heading px-3 mt-4 mb-1 text-muted">
|
|
<span>MAINTENANCE</span>
|
|
</h6>
|
|
<ul class="nav flex-column">
|
|
<li class="nav-item">
|
|
if isConfigPage {
|
|
<a class="nav-link" href="#" data-bs-toggle="collapse" data-bs-target="#configurationSubmenu" aria-expanded="true" aria-controls="configurationSubmenu">
|
|
<i class="fas fa-cogs me-2"></i>Configuration
|
|
<i class="fas fa-chevron-down ms-auto"></i>
|
|
</a>
|
|
} else {
|
|
<a class="nav-link collapsed" href="#" data-bs-toggle="collapse" data-bs-target="#configurationSubmenu" aria-expanded="false" aria-controls="configurationSubmenu">
|
|
<i class="fas fa-cogs me-2"></i>Configuration
|
|
<i class="fas fa-chevron-right ms-auto"></i>
|
|
</a>
|
|
}
|
|
if isConfigPage {
|
|
<div class="collapse show" id="configurationSubmenu">
|
|
<ul class="nav flex-column ms-3">
|
|
for _, menuItem := range GetConfigurationMenuItems() {
|
|
{{
|
|
isActiveItem := currentPath == menuItem.URL
|
|
}}
|
|
<li class="nav-item">
|
|
if isActiveItem {
|
|
<a class="nav-link py-2 active" href={templ.SafeURL(menuItem.URL)}>
|
|
<i class={menuItem.Icon + " me-2"}></i>{menuItem.Name}
|
|
</a>
|
|
} else {
|
|
<a class="nav-link py-2" href={templ.SafeURL(menuItem.URL)}>
|
|
<i class={menuItem.Icon + " me-2"}></i>{menuItem.Name}
|
|
</a>
|
|
}
|
|
</li>
|
|
}
|
|
</ul>
|
|
</div>
|
|
} else {
|
|
<div class="collapse" id="configurationSubmenu">
|
|
<ul class="nav flex-column ms-3">
|
|
for _, menuItem := range GetConfigurationMenuItems() {
|
|
<li class="nav-item">
|
|
<a class="nav-link py-2" href={templ.SafeURL(menuItem.URL)}>
|
|
<i class={menuItem.Icon + " me-2"}></i>{menuItem.Name}
|
|
</a>
|
|
</li>
|
|
}
|
|
</ul>
|
|
</div>
|
|
}
|
|
</li>
|
|
<li class="nav-item">
|
|
if currentPath == "/maintenance" {
|
|
<a class="nav-link active" href="/maintenance">
|
|
<i class="fas fa-list me-2"></i>Job Queue
|
|
</a>
|
|
} else {
|
|
<a class="nav-link" href="/maintenance">
|
|
<i class="fas fa-list me-2"></i>Job Queue
|
|
</a>
|
|
}
|
|
</li>
|
|
<li class="nav-item">
|
|
if currentPath == "/maintenance/workers" {
|
|
<a class="nav-link active" href="/maintenance/workers">
|
|
<i class="fas fa-user-cog me-2"></i>Workers
|
|
</a>
|
|
} else {
|
|
<a class="nav-link" href="/maintenance/workers">
|
|
<i class="fas fa-user-cog me-2"></i>Workers
|
|
</a>
|
|
}
|
|
</li>
|
|
</ul>
|
|
</div>
|
|
</div>
|
|
|
|
<!-- Main content -->
|
|
<main class="col-md-9 ms-sm-auto col-lg-10 px-md-4">
|
|
<div class="pt-3">
|
|
@content
|
|
</div>
|
|
</main>
|
|
</div>
|
|
</div>
|
|
|
|
<!-- Footer -->
|
|
<footer class="footer mt-auto py-3 bg-light">
|
|
<div class="container-fluid text-center">
|
|
<small class="text-muted">
|
|
© {fmt.Sprintf("%d", time.Now().Year())} SeaweedFS Admin v{version.VERSION_NUMBER}
|
|
if !strings.Contains(version.VERSION, "enterprise") {
|
|
<span class="mx-2">•</span>
|
|
<a href="https://seaweedfs.com" target="_blank" class="text-decoration-none">
|
|
<i class="fas fa-star me-1"></i>Enterprise Version Available
|
|
</a>
|
|
}
|
|
</small>
|
|
</div>
|
|
</footer>
|
|
|
|
<!-- Bootstrap JS -->
|
|
<script src="/static/js/bootstrap.bundle.min.js"></script>
|
|
<!-- Modal Alerts JS (replaces native alert/confirm) -->
|
|
<script src="/static/js/modal-alerts.js"></script>
|
|
<!-- Custom JS -->
|
|
<script src="/static/js/admin.js"></script>
|
|
<script src="/static/js/iam-utils.js"></script>
|
|
<script src="/static/js/s3tables.js"></script>
|
|
</body>
|
|
</html>
|
|
}
|
|
|
|
templ LoginForm(c *gin.Context, title string, errorMessage string) {
|
|
<!DOCTYPE html>
|
|
<html lang="en">
|
|
<head>
|
|
<meta charset="UTF-8">
|
|
<title>{title} - Login</title>
|
|
<link rel="icon" href="/static/favicon.ico" type="image/x-icon">
|
|
<meta name="viewport" content="width=device-width, initial-scale=1">
|
|
<link href="/static/css/bootstrap.min.css" rel="stylesheet">
|
|
<link href="/static/css/fontawesome.min.css" rel="stylesheet">
|
|
</head>
|
|
<body class="bg-light">
|
|
<div class="container">
|
|
<div class="row justify-content-center min-vh-100 align-items-center">
|
|
<div class="col-md-6 col-lg-4">
|
|
<div class="card shadow">
|
|
<div class="card-body p-5">
|
|
<div class="text-center mb-4">
|
|
<i class="fas fa-server fa-3x text-primary mb-3"></i>
|
|
<h4 class="card-title">{title}</h4>
|
|
<p class="text-muted">Please sign in to continue</p>
|
|
</div>
|
|
|
|
if errorMessage != "" {
|
|
<div class="alert alert-danger" role="alert">
|
|
<i class="fas fa-exclamation-triangle me-2"></i>
|
|
{errorMessage}
|
|
</div>
|
|
}
|
|
|
|
<form method="POST" action="/login">
|
|
<div class="mb-3">
|
|
<label for="username" class="form-label">Username</label>
|
|
<div class="input-group">
|
|
<span class="input-group-text">
|
|
<i class="fas fa-user"></i>
|
|
</span>
|
|
<input type="text" class="form-control" id="username" name="username" required>
|
|
</div>
|
|
</div>
|
|
|
|
<div class="mb-4">
|
|
<label for="password" class="form-label">Password</label>
|
|
<div class="input-group">
|
|
<span class="input-group-text">
|
|
<i class="fas fa-lock"></i>
|
|
</span>
|
|
<input type="password" class="form-control" id="password" name="password" required>
|
|
</div>
|
|
</div>
|
|
|
|
<button type="submit" class="btn btn-primary w-100">
|
|
<i class="fas fa-sign-in-alt me-2"></i>Sign In
|
|
</button>
|
|
</form>
|
|
</div>
|
|
</div>
|
|
</div>
|
|
</div>
|
|
</div>
|
|
|
|
<script src="/static/js/bootstrap.bundle.min.js"></script>
|
|
</body>
|
|
</html>
|
|
}
|