2bb21ea276
* feat: Add Iceberg REST Catalog server Implement Iceberg REST Catalog API on a separate port (default 8181) that exposes S3 Tables metadata through the Apache Iceberg REST protocol. - Add new weed/s3api/iceberg package with REST handlers - Implement /v1/config endpoint returning catalog configuration - Implement namespace endpoints (list/create/get/head/delete) - Implement table endpoints (list/create/load/head/delete/update) - Add -port.iceberg flag to S3 standalone server (s3.go) - Add -s3.port.iceberg flag to combined server mode (server.go) - Add -s3.port.iceberg flag to mini cluster mode (mini.go) - Support prefix-based routing for multiple catalogs The Iceberg REST server reuses S3 Tables metadata storage under /table-buckets and enables DuckDB, Spark, and other Iceberg clients to connect to SeaweedFS as a catalog. * feat: Add Iceberg Catalog pages to admin UI Add admin UI pages to browse Iceberg catalogs, namespaces, and tables. - Add Iceberg Catalog menu item under Object Store navigation - Create iceberg_catalog.templ showing catalog overview with REST info - Create iceberg_namespaces.templ listing namespaces in a catalog - Create iceberg_tables.templ listing tables in a namespace - Add handlers and routes in admin_handlers.go - Add Iceberg data provider methods in s3tables_management.go - Add Iceberg data types in types.go The Iceberg Catalog pages provide visibility into the same S3 Tables data through an Iceberg-centric lens, including REST endpoint examples for DuckDB and PyIceberg. * test: Add Iceberg catalog integration tests and reorg s3tables tests - Reorganize existing s3tables tests to test/s3tables/table-buckets/ - Add new test/s3tables/catalog/ for Iceberg REST catalog tests - Add TestIcebergConfig to verify /v1/config endpoint - Add TestIcebergNamespaces to verify namespace listing - Add TestDuckDBIntegration for DuckDB connectivity (requires Docker) - Update CI workflow to use new test paths * fix: Generate proper random UUIDs for Iceberg tables Address code review feedback: - Replace placeholder UUID with crypto/rand-based UUID v4 generation - Add detailed TODO comments for handleUpdateTable stub explaining the required atomic metadata swap implementation * fix: Serve Iceberg on localhost listener when binding to different interface Address code review feedback: properly serve the localhost listener when the Iceberg server is bound to a non-localhost interface. * ci: Add Iceberg catalog integration tests to CI Add new job to run Iceberg catalog tests in CI, along with: - Iceberg package build verification - Iceberg unit tests - Iceberg go vet checks - Iceberg format checks * fix: Address code review feedback for Iceberg implementation - fix: Replace hardcoded account ID with s3_constants.AccountAdminId in buildTableBucketARN() - fix: Improve UUID generation error handling with deterministic fallback (timestamp + PID + counter) - fix: Update handleUpdateTable to return HTTP 501 Not Implemented instead of fake success - fix: Better error handling in handleNamespaceExists to distinguish 404 from 500 errors - fix: Use relative URL in template instead of hardcoded localhost:8181 - fix: Add HTTP timeout to test's waitForService function to avoid hangs - fix: Use dynamic ephemeral ports in integration tests to avoid flaky parallel failures - fix: Add Iceberg port to final port configuration logging in mini.go * fix: Address critical issues in Iceberg implementation - fix: Cache table UUIDs to ensure persistence across LoadTable calls The UUID now remains stable for the lifetime of the server session. TODO: For production, UUIDs should be persisted in S3 Tables metadata. - fix: Remove redundant URL-encoded namespace parsing mux router already decodes %1F to \x1F before passing to handlers. Redundant ReplaceAll call could cause bugs with literal %1F in namespace. * fix: Improve test robustness and reduce code duplication - fix: Make DuckDB test more robust by failing on unexpected errors Instead of silently logging errors, now explicitly check for expected conditions (extension not available) and skip the test appropriately. - fix: Extract username helper method to reduce duplication Created getUsername() helper in AdminHandlers to avoid duplicating the username retrieval logic across Iceberg page handlers. * fix: Add mutex protection to table UUID cache Protects concurrent access to the tableUUIDs map with sync.RWMutex. Uses read-lock for fast path when UUID already cached, and write-lock for generating new UUIDs. Includes double-check pattern to handle race condition between read-unlock and write-lock. * style: fix go fmt errors * feat(iceberg): persist table UUID in S3 Tables metadata * feat(admin): configure Iceberg port in Admin UI and commands * refactor: address review comments (flags, tests, handlers) - command/mini: fix tracking of explicit s3.port.iceberg flag - command/admin: add explicit -iceberg.port flag - admin/handlers: reuse getUsername helper - tests: use 127.0.0.1 for ephemeral ports and os.Stat for file size check * test: check error from FileStat in verify_gc_empty_test
247 lines
14 KiB
Go
247 lines
14 KiB
Go
// Code generated by templ - DO NOT EDIT.
|
|
|
|
// templ: version: v0.3.960
|
|
package app
|
|
|
|
//lint:file-ignore SA4006 This context is only used if a nested component is present.
|
|
|
|
import "github.com/a-h/templ"
|
|
import templruntime "github.com/a-h/templ/runtime"
|
|
|
|
import "fmt"
|
|
import "github.com/seaweedfs/seaweedfs/weed/admin/dash"
|
|
|
|
func Subscribers(data dash.SubscribersData) templ.Component {
|
|
return templruntime.GeneratedTemplate(func(templ_7745c5c3_Input templruntime.GeneratedComponentInput) (templ_7745c5c3_Err error) {
|
|
templ_7745c5c3_W, ctx := templ_7745c5c3_Input.Writer, templ_7745c5c3_Input.Context
|
|
if templ_7745c5c3_CtxErr := ctx.Err(); templ_7745c5c3_CtxErr != nil {
|
|
return templ_7745c5c3_CtxErr
|
|
}
|
|
templ_7745c5c3_Buffer, templ_7745c5c3_IsBuffer := templruntime.GetBuffer(templ_7745c5c3_W)
|
|
if !templ_7745c5c3_IsBuffer {
|
|
defer func() {
|
|
templ_7745c5c3_BufErr := templruntime.ReleaseBuffer(templ_7745c5c3_Buffer)
|
|
if templ_7745c5c3_Err == nil {
|
|
templ_7745c5c3_Err = templ_7745c5c3_BufErr
|
|
}
|
|
}()
|
|
}
|
|
ctx = templ.InitializeContext(ctx)
|
|
templ_7745c5c3_Var1 := templ.GetChildren(ctx)
|
|
if templ_7745c5c3_Var1 == nil {
|
|
templ_7745c5c3_Var1 = templ.NopComponent
|
|
}
|
|
ctx = templ.ClearChildren(ctx)
|
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 1, "<div class=\"container-fluid\"><div class=\"row\"><div class=\"col-12\"><div class=\"d-flex justify-content-between align-items-center mb-4\"><h1 class=\"h3 mb-0\">Message Queue Subscribers</h1><small class=\"text-muted\">Last updated: ")
|
|
if templ_7745c5c3_Err != nil {
|
|
return templ_7745c5c3_Err
|
|
}
|
|
var templ_7745c5c3_Var2 string
|
|
templ_7745c5c3_Var2, templ_7745c5c3_Err = templ.JoinStringErrs(data.LastUpdated.Format("2006-01-02 15:04:05"))
|
|
if templ_7745c5c3_Err != nil {
|
|
return templ.Error{Err: templ_7745c5c3_Err, FileName: `weed/admin/view/app/subscribers.templ`, Line: 12, Col: 107}
|
|
}
|
|
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var2))
|
|
if templ_7745c5c3_Err != nil {
|
|
return templ_7745c5c3_Err
|
|
}
|
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 2, "</small></div><!-- Summary Cards --><div class=\"row mb-4\"><div class=\"col-md-4\"><div class=\"card text-center\"><div class=\"card-body\"><h5 class=\"card-title\">Total Subscribers</h5><h3 class=\"text-primary\">")
|
|
if templ_7745c5c3_Err != nil {
|
|
return templ_7745c5c3_Err
|
|
}
|
|
var templ_7745c5c3_Var3 string
|
|
templ_7745c5c3_Var3, templ_7745c5c3_Err = templ.JoinStringErrs(fmt.Sprintf("%d", data.TotalSubscribers))
|
|
if templ_7745c5c3_Err != nil {
|
|
return templ.Error{Err: templ_7745c5c3_Err, FileName: `weed/admin/view/app/subscribers.templ`, Line: 21, Col: 98}
|
|
}
|
|
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var3))
|
|
if templ_7745c5c3_Err != nil {
|
|
return templ_7745c5c3_Err
|
|
}
|
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 3, "</h3></div></div></div><div class=\"col-md-4\"><div class=\"card text-center\"><div class=\"card-body\"><h5 class=\"card-title\">Active Subscribers</h5><h3 class=\"text-success\">")
|
|
if templ_7745c5c3_Err != nil {
|
|
return templ_7745c5c3_Err
|
|
}
|
|
var templ_7745c5c3_Var4 string
|
|
templ_7745c5c3_Var4, templ_7745c5c3_Err = templ.JoinStringErrs(fmt.Sprintf("%d", data.ActiveSubscribers))
|
|
if templ_7745c5c3_Err != nil {
|
|
return templ.Error{Err: templ_7745c5c3_Err, FileName: `weed/admin/view/app/subscribers.templ`, Line: 29, Col: 99}
|
|
}
|
|
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var4))
|
|
if templ_7745c5c3_Err != nil {
|
|
return templ_7745c5c3_Err
|
|
}
|
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 4, "</h3></div></div></div><div class=\"col-md-4\"><div class=\"card text-center\"><div class=\"card-body\"><h5 class=\"card-title\">Inactive Subscribers</h5><h3 class=\"text-warning\">")
|
|
if templ_7745c5c3_Err != nil {
|
|
return templ_7745c5c3_Err
|
|
}
|
|
var templ_7745c5c3_Var5 string
|
|
templ_7745c5c3_Var5, templ_7745c5c3_Err = templ.JoinStringErrs(fmt.Sprintf("%d", data.TotalSubscribers-data.ActiveSubscribers))
|
|
if templ_7745c5c3_Err != nil {
|
|
return templ.Error{Err: templ_7745c5c3_Err, FileName: `weed/admin/view/app/subscribers.templ`, Line: 37, Col: 123}
|
|
}
|
|
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var5))
|
|
if templ_7745c5c3_Err != nil {
|
|
return templ_7745c5c3_Err
|
|
}
|
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 5, "</h3></div></div></div></div><!-- Subscribers Table --><div class=\"card\"><div class=\"card-header d-flex justify-content-between align-items-center\"><h5 class=\"mb-0\">Subscribers</h5><div><button class=\"btn btn-sm btn-outline-secondary\" onclick=\"exportSubscribersCSV()\"><i class=\"fas fa-download me-1\"></i>Export CSV</button></div></div><div class=\"card-body\">")
|
|
if templ_7745c5c3_Err != nil {
|
|
return templ_7745c5c3_Err
|
|
}
|
|
if len(data.Subscribers) == 0 {
|
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 6, "<div class=\"text-center py-4\"><i class=\"fas fa-user-friends fa-3x text-muted mb-3\"></i><h5>No Subscribers Found</h5><p class=\"text-muted\">No message queue subscribers are currently active.</p></div>")
|
|
if templ_7745c5c3_Err != nil {
|
|
return templ_7745c5c3_Err
|
|
}
|
|
} else {
|
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 7, "<div class=\"table-responsive\"><table class=\"table table-striped\" id=\"subscribersTable\"><thead><tr><th>Subscriber Name</th><th>Topic</th><th>Consumer Group</th><th>Status</th><th>Messages Processed</th><th>Last Seen</th><th>Created</th></tr></thead> <tbody>")
|
|
if templ_7745c5c3_Err != nil {
|
|
return templ_7745c5c3_Err
|
|
}
|
|
for _, subscriber := range data.Subscribers {
|
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 8, "<tr><td><strong>")
|
|
if templ_7745c5c3_Err != nil {
|
|
return templ_7745c5c3_Err
|
|
}
|
|
var templ_7745c5c3_Var6 string
|
|
templ_7745c5c3_Var6, templ_7745c5c3_Err = templ.JoinStringErrs(subscriber.Name)
|
|
if templ_7745c5c3_Err != nil {
|
|
return templ.Error{Err: templ_7745c5c3_Err, FileName: `weed/admin/view/app/subscribers.templ`, Line: 78, Col: 76}
|
|
}
|
|
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var6))
|
|
if templ_7745c5c3_Err != nil {
|
|
return templ_7745c5c3_Err
|
|
}
|
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 9, "</strong></td><td><span class=\"badge bg-info\">")
|
|
if templ_7745c5c3_Err != nil {
|
|
return templ_7745c5c3_Err
|
|
}
|
|
var templ_7745c5c3_Var7 string
|
|
templ_7745c5c3_Var7, templ_7745c5c3_Err = templ.JoinStringErrs(subscriber.Topic)
|
|
if templ_7745c5c3_Err != nil {
|
|
return templ.Error{Err: templ_7745c5c3_Err, FileName: `weed/admin/view/app/subscribers.templ`, Line: 81, Col: 97}
|
|
}
|
|
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var7))
|
|
if templ_7745c5c3_Err != nil {
|
|
return templ_7745c5c3_Err
|
|
}
|
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 10, "</span></td><td>")
|
|
if templ_7745c5c3_Err != nil {
|
|
return templ_7745c5c3_Err
|
|
}
|
|
var templ_7745c5c3_Var8 string
|
|
templ_7745c5c3_Var8, templ_7745c5c3_Err = templ.JoinStringErrs(subscriber.ConsumerGroup)
|
|
if templ_7745c5c3_Err != nil {
|
|
return templ.Error{Err: templ_7745c5c3_Err, FileName: `weed/admin/view/app/subscribers.templ`, Line: 83, Col: 77}
|
|
}
|
|
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var8))
|
|
if templ_7745c5c3_Err != nil {
|
|
return templ_7745c5c3_Err
|
|
}
|
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 11, "</td><td>")
|
|
if templ_7745c5c3_Err != nil {
|
|
return templ_7745c5c3_Err
|
|
}
|
|
if subscriber.Status == "active" {
|
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 12, "<span class=\"badge bg-success\">Active</span>")
|
|
if templ_7745c5c3_Err != nil {
|
|
return templ_7745c5c3_Err
|
|
}
|
|
} else if subscriber.Status == "inactive" {
|
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 13, "<span class=\"badge bg-warning\">Inactive</span>")
|
|
if templ_7745c5c3_Err != nil {
|
|
return templ_7745c5c3_Err
|
|
}
|
|
} else {
|
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 14, "<span class=\"badge bg-secondary\">")
|
|
if templ_7745c5c3_Err != nil {
|
|
return templ_7745c5c3_Err
|
|
}
|
|
var templ_7745c5c3_Var9 string
|
|
templ_7745c5c3_Var9, templ_7745c5c3_Err = templ.JoinStringErrs(subscriber.Status)
|
|
if templ_7745c5c3_Err != nil {
|
|
return templ.Error{Err: templ_7745c5c3_Err, FileName: `weed/admin/view/app/subscribers.templ`, Line: 90, Col: 107}
|
|
}
|
|
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var9))
|
|
if templ_7745c5c3_Err != nil {
|
|
return templ_7745c5c3_Err
|
|
}
|
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 15, "</span>")
|
|
if templ_7745c5c3_Err != nil {
|
|
return templ_7745c5c3_Err
|
|
}
|
|
}
|
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 16, "</td><td>")
|
|
if templ_7745c5c3_Err != nil {
|
|
return templ_7745c5c3_Err
|
|
}
|
|
var templ_7745c5c3_Var10 string
|
|
templ_7745c5c3_Var10, templ_7745c5c3_Err = templ.JoinStringErrs(fmt.Sprintf("%d", subscriber.MessageCount))
|
|
if templ_7745c5c3_Err != nil {
|
|
return templ.Error{Err: templ_7745c5c3_Err, FileName: `weed/admin/view/app/subscribers.templ`, Line: 93, Col: 95}
|
|
}
|
|
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var10))
|
|
if templ_7745c5c3_Err != nil {
|
|
return templ_7745c5c3_Err
|
|
}
|
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 17, "</td><td>")
|
|
if templ_7745c5c3_Err != nil {
|
|
return templ_7745c5c3_Err
|
|
}
|
|
if !subscriber.LastSeen.IsZero() {
|
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 18, "<span class=\"text-muted\">")
|
|
if templ_7745c5c3_Err != nil {
|
|
return templ_7745c5c3_Err
|
|
}
|
|
var templ_7745c5c3_Var11 string
|
|
templ_7745c5c3_Var11, templ_7745c5c3_Err = templ.JoinStringErrs(subscriber.LastSeen.Format("2006-01-02 15:04:05"))
|
|
if templ_7745c5c3_Err != nil {
|
|
return templ.Error{Err: templ_7745c5c3_Err, FileName: `weed/admin/view/app/subscribers.templ`, Line: 96, Col: 131}
|
|
}
|
|
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var11))
|
|
if templ_7745c5c3_Err != nil {
|
|
return templ_7745c5c3_Err
|
|
}
|
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 19, "</span>")
|
|
if templ_7745c5c3_Err != nil {
|
|
return templ_7745c5c3_Err
|
|
}
|
|
} else {
|
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 20, "<span class=\"text-muted\">Never</span>")
|
|
if templ_7745c5c3_Err != nil {
|
|
return templ_7745c5c3_Err
|
|
}
|
|
}
|
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 21, "</td><td><span class=\"text-muted\">")
|
|
if templ_7745c5c3_Err != nil {
|
|
return templ_7745c5c3_Err
|
|
}
|
|
var templ_7745c5c3_Var12 string
|
|
templ_7745c5c3_Var12, templ_7745c5c3_Err = templ.JoinStringErrs(subscriber.CreatedAt.Format("2006-01-02 15:04:05"))
|
|
if templ_7745c5c3_Err != nil {
|
|
return templ.Error{Err: templ_7745c5c3_Err, FileName: `weed/admin/view/app/subscribers.templ`, Line: 102, Col: 128}
|
|
}
|
|
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var12))
|
|
if templ_7745c5c3_Err != nil {
|
|
return templ_7745c5c3_Err
|
|
}
|
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 22, "</span></td></tr>")
|
|
if templ_7745c5c3_Err != nil {
|
|
return templ_7745c5c3_Err
|
|
}
|
|
}
|
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 23, "</tbody></table></div>")
|
|
if templ_7745c5c3_Err != nil {
|
|
return templ_7745c5c3_Err
|
|
}
|
|
}
|
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 24, "</div></div></div></div></div><script>\n function exportSubscribersCSV() {\n const table = document.getElementById('subscribersTable');\n if (!table) return;\n \n let csv = 'Subscriber Name,Topic,Consumer Group,Status,Messages Processed,Last Seen,Created\\n';\n \n const rows = table.querySelectorAll('tbody tr');\n rows.forEach(row => {\n const cells = row.querySelectorAll('td');\n if (cells.length >= 7) {\n const rowData = [\n cells[0].querySelector('strong')?.textContent || '',\n cells[1].querySelector('.badge')?.textContent || '',\n cells[2].textContent || '',\n cells[3].querySelector('.badge')?.textContent || '',\n cells[4].textContent || '',\n cells[5].querySelector('span')?.textContent || '',\n cells[6].querySelector('span')?.textContent || ''\n ];\n csv += rowData.map(field => `\"${field.replace(/\"/g, '\"\"')}\"`).join(',') + '\\n';\n }\n });\n \n const blob = new Blob([csv], { type: 'text/csv;charset=utf-8;' });\n const link = document.createElement('a');\n const url = URL.createObjectURL(blob);\n link.setAttribute('href', url);\n link.setAttribute('download', 'subscribers.csv');\n link.style.visibility = 'hidden';\n document.body.appendChild(link);\n link.click();\n document.body.removeChild(link);\n }\n </script>")
|
|
if templ_7745c5c3_Err != nil {
|
|
return templ_7745c5c3_Err
|
|
}
|
|
return nil
|
|
})
|
|
}
|
|
|
|
var _ = templruntime.GeneratedTemplate
|