Commit Graph

22 Commits

Author SHA1 Message Date
Chris Lu
7b08cf74ed consistent template generation 2026-02-22 13:34:06 -08:00
Chris Lu
5a0204310c Add Iceberg admin UI (#8246)
* Add Iceberg table details view

* Enhance Iceberg catalog browsing UI

* Fix Iceberg UI security and logic issues

- Fix selectSchema() and partitionFieldsFromFullMetadata() to always search for matching IDs instead of checking != 0
- Fix snapshotsFromFullMetadata() to defensive-copy before sorting to prevent mutating caller's slice
- Fix XSS vulnerabilities in s3tables.js: replace innerHTML with textContent/createElement for user-controlled data
- Fix deleteIcebergTable() to redirect to namespace tables list on details page instead of reloading
- Fix data-bs-target in iceberg_namespaces.templ: remove templ.SafeURL for CSS selector
- Add catalogName to delete modal data attributes for proper redirect
- Remove unused hidden inputs from create table form (icebergTableBucketArn, icebergTableNamespace)

* Regenerate templ files for Iceberg UI updates

* Support complex Iceberg type objects in schema

Change Type field from string to json.RawMessage in both IcebergSchemaFieldInfo
and internal icebergSchemaField to properly handle Iceberg spec's complex type
objects (e.g. {"type": "struct", "fields": [...]}). Currently test data
only shows primitive string types, but this change makes the implementation
defensively robust for future complex types by preserving the exact JSON
representation. Add typeToString() helper and update schema extraction
functions to marshal string types as JSON. Update template to convert
json.RawMessage to string for display.

* Regenerate templ files for Type field changes

* templ

* Fix additional Iceberg UI issues from code review

- Fix lazy-load flag that was set before async operation completed, preventing retries
  on error; now sets loaded flag only after successful load and throws error to caller
  for proper error handling and UI updates
- Add zero-time guards for CreatedAt and ModifiedAt fields in table details to avoid
  displaying Go zero-time values; render dash when time is zero
- Add URL path escaping for all catalog/namespace/table names in URLs to prevent
  malformed URLs when names contain special characters like /, ?, or #
- Remove redundant innerHTML clear in loadIcebergNamespaceTables that cleared twice
  before appending the table list
- Fix selectSnapshotForMetrics to remove != 0 guard for consistency with selectSchema
  fix; now always searches for CurrentSnapshotID without zero-value gate
- Enhance typeToString() helper to display '(complex)' for non-primitive JSON types

* Regenerate templ files for Phase 3 updates

* Fix template generation to use correct file paths

Run templ generate from repo root instead of weed/admin directory to ensure
generated _templ.go files have correct absolute paths in error messages
(e.g., 'weed/admin/view/app/iceberg_table_details.templ' instead of
'app/iceberg_table_details.templ'). This ensures both 'make admin-generate'
at repo root and 'make generate' in weed/admin directory produce identical
output with consistent file path references.

* Regenerate template files with correct path references

* Validate S3 Tables names in UI

- Add client-side validation for table bucket and namespace names to surface
  errors for invalid characters (dots/underscores) before submission
- Use HTML validity messages with reportValidity for immediate feedback
- Update namespace helper text to reflect actual constraints (single-level,
  lowercase letters, numbers, and underscores)

* Regenerate templ files for namespace helper text

* Fix Iceberg catalog REST link and actions

* Disallow S3 object access on table buckets

* Validate Iceberg layout for table bucket objects

* Fix REST API link to /v1/config

* merge iceberg page with table bucket page

* Allowed Trino/Iceberg stats files in metadata validation

* fixes

  - Backend/data handling:
      - Normalized Iceberg type display and fallback handling in weed/admin/dash/s3tables_management.go.
      - Fixed snapshot fallback pointer semantics in weed/admin/dash/s3tables_management.go.
      - Added CSRF token generation/propagation/validation for namespace create/delete in:
          - weed/admin/dash/csrf.go
          - weed/admin/dash/auth_middleware.go
          - weed/admin/dash/middleware.go
          - weed/admin/dash/s3tables_management.go
          - weed/admin/view/layout/layout.templ
          - weed/admin/static/js/s3tables.js
  - UI/template fixes:
      - Zero-time guards for CreatedAt fields in:
          - weed/admin/view/app/iceberg_namespaces.templ
          - weed/admin/view/app/iceberg_tables.templ
      - Fixed invalid templ-in-script interpolation and host/port rendering in:
          - weed/admin/view/app/iceberg_catalog.templ
          - weed/admin/view/app/s3tables_buckets.templ
      - Added data-catalog-name consistency on Iceberg delete action in weed/admin/view/app/iceberg_tables.templ.
      - Updated retry wording in weed/admin/static/js/s3tables.js.
      - Regenerated all affected _templ.go files.
  - S3 API/comment follow-ups:
      - Reused cached table-bucket validator in weed/s3api/bucket_paths.go.
      - Added validation-failure debug logging in weed/s3api/s3api_object_handlers_tagging.go.
      - Added multipart path-validation design comment in weed/s3api/s3api_object_handlers_multipart.go.
  - Build tooling:
      - Fixed templ generate working directory issues in weed/admin/Makefile (watch + pattern rule).

* populate data

* test/s3tables: harden populate service checks

* admin: skip table buckets in object-store bucket list

* admin sidebar: move object store to top-level links

* admin iceberg catalog: guard zero times and escape links

* admin forms: add csrf/error handling and client-side name validation

* admin s3tables: fix namespace delete modal redeclaration

* admin: replace native confirm dialogs with modal helpers

* admin modal-alerts: remove noisy confirm usage console log

* reduce logs

* test/s3tables: use partitioned tables in trino and spark populate

* admin file browser: normalize filer ServerAddress for HTTP parsing
2026-02-08 20:06:32 -08:00
Chris Lu
72a8f598f2 Fix Maintenance Task Sorting and Refactor Log Persistence (#8199)
* fix float stepping

* do not auto refresh

* only logs when non 200 status

* fix maintenance task sorting and cleanup redundant handler logic

* Refactor log retrieval to persist to disk and fix slowness

- Move log retrieval to disk-based persistence in GetMaintenanceTaskDetail
- Implement background log fetching on task completion in worker_grpc_server.go
- Implement async background refresh for in-progress tasks
- Completely remove blocking gRPC calls from the UI path to fix 10s timeouts
- Cleanup debug logs and performance profiling code

* Ensure consistent deterministic sorting in config_persistence cleanup

* Replace magic numbers with constants and remove debug logs

- Added descriptive constants for truncation limits and timeouts in admin_server.go and worker_grpc_server.go
- Replaced magic numbers with these constants throughout the codebase
- Verified removal of stdout debug printing
- Ensured consistent truncation logic during log persistence

* Address code review feedback on history truncation and logging logic

- Fix AssignmentHistory double-serialization by copying task in GetMaintenanceTaskDetail
- Fix handleTaskCompletion logging logic (mutually exclusive success/failure logs)
- Remove unused Timeout field from LogRequestContext and sync select timeouts with constants
- Ensure AssignmentHistory is only provided in the top-level field for better JSON structure

* Implement goroutine leak protection and request deduplication

- Add request deduplication in RequestTaskLogs to prevent multiple concurrent fetches for the same task
- Implement safe cleanup in timeout handlers to avoid race conditions in pendingLogRequests map
- Add a 10s cooldown for background log refreshes in GetMaintenanceTaskDetail to prevent spamming
- Ensure all persistent log-fetching goroutines are bounded and efficiently managed

* Fix potential nil pointer panics in maintenance handlers

- Add nil checks for adminServer in ShowTaskDetail, ShowMaintenanceWorkers, and UpdateTaskConfig
- Update getMaintenanceQueueData to return a descriptive error instead of nil when adminServer is uninitialized
- Ensure internal helper methods consistently check for adminServer initialization before use

* Strictly enforce disk-only log reading

- Remove background log fetching from GetMaintenanceTaskDetail to prevent timeouts and network calls during page view
- Remove unused lastLogFetch tracking fields to clean up dead code
- Ensure logs are only updated upon task completion via handleTaskCompletion

* Refactor GetWorkerLogs to read from disk

- Update /api/maintenance/workers/:id/logs endpoint to use configPersistence.LoadTaskExecutionLogs
- Remove synchronous gRPC call RequestTaskLogs to prevent timeouts and bad gateway errors
- Ensure consistent log retrieval behavior across the application (disk-only)

* Fix timestamp parsing in log viewer

- Update task_detail.templ JS to handle both ISO 8601 strings and Unix timestamps
- Fix "Invalid time value" error when displaying logs fetched from disk
- Regenerate templates

* master: fallback to HDD if SSD volumes are full in Assign

* worker: improve EC detection logging and fix skip counters

* worker: add Sync method to TaskLogger interface

* worker: implement Sync and ensure logs are flushed before task completion

* admin: improve task log retrieval with retries and better timeouts

* admin: robust timestamp parsing in task detail view
2026-02-04 08:48:55 -08:00
Chris Lu
1c1f80358f templ 2026-02-03 00:10:43 -08:00
Chris Lu
2bb21ea276 feat: Add Iceberg REST Catalog server and admin UI (#8175)
* feat: Add Iceberg REST Catalog server

Implement Iceberg REST Catalog API on a separate port (default 8181)
that exposes S3 Tables metadata through the Apache Iceberg REST protocol.

- Add new weed/s3api/iceberg package with REST handlers
- Implement /v1/config endpoint returning catalog configuration
- Implement namespace endpoints (list/create/get/head/delete)
- Implement table endpoints (list/create/load/head/delete/update)
- Add -port.iceberg flag to S3 standalone server (s3.go)
- Add -s3.port.iceberg flag to combined server mode (server.go)
- Add -s3.port.iceberg flag to mini cluster mode (mini.go)
- Support prefix-based routing for multiple catalogs

The Iceberg REST server reuses S3 Tables metadata storage under
/table-buckets and enables DuckDB, Spark, and other Iceberg clients
to connect to SeaweedFS as a catalog.

* feat: Add Iceberg Catalog pages to admin UI

Add admin UI pages to browse Iceberg catalogs, namespaces, and tables.

- Add Iceberg Catalog menu item under Object Store navigation
- Create iceberg_catalog.templ showing catalog overview with REST info
- Create iceberg_namespaces.templ listing namespaces in a catalog
- Create iceberg_tables.templ listing tables in a namespace
- Add handlers and routes in admin_handlers.go
- Add Iceberg data provider methods in s3tables_management.go
- Add Iceberg data types in types.go

The Iceberg Catalog pages provide visibility into the same S3 Tables
data through an Iceberg-centric lens, including REST endpoint examples
for DuckDB and PyIceberg.

* test: Add Iceberg catalog integration tests and reorg s3tables tests

- Reorganize existing s3tables tests to test/s3tables/table-buckets/
- Add new test/s3tables/catalog/ for Iceberg REST catalog tests
- Add TestIcebergConfig to verify /v1/config endpoint
- Add TestIcebergNamespaces to verify namespace listing
- Add TestDuckDBIntegration for DuckDB connectivity (requires Docker)
- Update CI workflow to use new test paths

* fix: Generate proper random UUIDs for Iceberg tables

Address code review feedback:
- Replace placeholder UUID with crypto/rand-based UUID v4 generation
- Add detailed TODO comments for handleUpdateTable stub explaining
  the required atomic metadata swap implementation

* fix: Serve Iceberg on localhost listener when binding to different interface

Address code review feedback: properly serve the localhost listener
when the Iceberg server is bound to a non-localhost interface.

* ci: Add Iceberg catalog integration tests to CI

Add new job to run Iceberg catalog tests in CI, along with:
- Iceberg package build verification
- Iceberg unit tests
- Iceberg go vet checks
- Iceberg format checks

* fix: Address code review feedback for Iceberg implementation

- fix: Replace hardcoded account ID with s3_constants.AccountAdminId in buildTableBucketARN()
- fix: Improve UUID generation error handling with deterministic fallback (timestamp + PID + counter)
- fix: Update handleUpdateTable to return HTTP 501 Not Implemented instead of fake success
- fix: Better error handling in handleNamespaceExists to distinguish 404 from 500 errors
- fix: Use relative URL in template instead of hardcoded localhost:8181
- fix: Add HTTP timeout to test's waitForService function to avoid hangs
- fix: Use dynamic ephemeral ports in integration tests to avoid flaky parallel failures
- fix: Add Iceberg port to final port configuration logging in mini.go

* fix: Address critical issues in Iceberg implementation

- fix: Cache table UUIDs to ensure persistence across LoadTable calls
  The UUID now remains stable for the lifetime of the server session.
  TODO: For production, UUIDs should be persisted in S3 Tables metadata.

- fix: Remove redundant URL-encoded namespace parsing
  mux router already decodes %1F to \x1F before passing to handlers.
  Redundant ReplaceAll call could cause bugs with literal %1F in namespace.

* fix: Improve test robustness and reduce code duplication

- fix: Make DuckDB test more robust by failing on unexpected errors
  Instead of silently logging errors, now explicitly check for expected
  conditions (extension not available) and skip the test appropriately.

- fix: Extract username helper method to reduce duplication
  Created getUsername() helper in AdminHandlers to avoid duplicating
  the username retrieval logic across Iceberg page handlers.

* fix: Add mutex protection to table UUID cache

Protects concurrent access to the tableUUIDs map with sync.RWMutex.
Uses read-lock for fast path when UUID already cached, and write-lock
for generating new UUIDs. Includes double-check pattern to handle race
condition between read-unlock and write-lock.

* style: fix go fmt errors

* feat(iceberg): persist table UUID in S3 Tables metadata

* feat(admin): configure Iceberg port in Admin UI and commands

* refactor: address review comments (flags, tests, handlers)

- command/mini: fix tracking of explicit s3.port.iceberg flag
- command/admin: add explicit -iceberg.port flag
- admin/handlers: reuse getUsername helper
- tests: use 127.0.0.1 for ephemeral ports and os.Stat for file size check

* test: check error from FileStat in verify_gc_empty_test
2026-02-02 23:12:13 -08:00
Chris Lu
3b9e367c1a templ 2026-02-02 20:34:37 -08:00
Chris Lu
79722bcf30 Add s3tables shell and admin UI (#8172)
* Add shared s3tables manager

* Add s3tables shell commands

* Add s3tables admin API

* Add s3tables admin UI

* Fix admin s3tables namespace create

* Rename table buckets menu

* Centralize s3tables tag validation

* Reuse s3tables manager in admin

* Extract s3tables list limit

* Add s3tables bucket ARN helper

* Remove write middleware from s3tables APIs

* Fix bucket link and policy hint

* Fix table tag parsing and nav link

* Disable namespace table link on invalid ARN

* Improve s3tables error decode

* Return flag parse errors for s3tables tag

* Accept query params for namespace create

* Bind namespace create form data

* Read s3tables JS data from DOM

* s3tables: allow empty region ARN

* shell: pass s3tables account id

* shell: require account for table buckets

* shell: use bucket name for namespaces

* shell: use bucket name for tables

* shell: use bucket name for tags

* admin: add table buckets links in file browser

* s3api: reuse s3tables tag validation

* admin: harden s3tables UI handlers

* fix admin list table buckets

* allow admin s3tables access

* validate s3tables bucket tags

* log s3tables bucket metadata errors

* rollback table bucket on owner failure

* show s3tables bucket owner

* add s3tables iam conditions

* Add s3tables user permissions UI

* Authorize s3tables using identity actions

* Add s3tables permissions to user modal

* Disambiguate bucket scope in user permissions

* Block table bucket names that match S3 buckets

* Pretty-print IAM identity JSON

* Include tags in s3tables permission context

* admin: refactor S3 Tables inline JavaScript into a separate file

* s3tables: extend IAM policy condition operators support

* shell: use LookupEntry wrapper for s3tables bucket conflict check

* admin: handle buildBucketPermissions validation in create/update flows
2026-01-30 22:57:05 -08:00
Chris Lu
c5b53397c6 templ 2026-01-26 11:38:18 -08:00
Chris Lu
5ba0db7af4 Merge branch 'origin/master' into master
Resolved merge conflicts in:
- weed/admin/static/js/modal-alerts.js: Adopted incoming improvements and HTML support.
- weed/admin/view/app/collection_details.templ: Switched to showAlert info type.
- weed/admin/view/app/file_browser.templ: Used descriptive delete message.
- weed/admin/view/app/maintenance_workers.templ: Used encoding and headers in pauseWorker.
- weed/admin/view/app/object_store_users.templ: Restored accidentally deleted delete functions and used encodeURIComponent.
- weed/admin/view/app/policies.templ: Standardized on showAlert and descriptive confirmations.

Regenerated all templ files.
2026-01-26 11:37:12 -08:00
Chris Lu
5a7c74feac migrate IAM policies to multi-file storage (#8114)
* Add IAM gRPC service definition

- Add GetConfiguration/PutConfiguration for config management
- Add CreateUser/GetUser/UpdateUser/DeleteUser/ListUsers for user management
- Add CreateAccessKey/DeleteAccessKey/GetUserByAccessKey for access key management
- Methods mirror existing IAM HTTP API functionality

* Add IAM gRPC handlers on filer server

- Implement IamGrpcServer with CredentialManager integration
- Handle configuration get/put operations
- Handle user CRUD operations
- Handle access key create/delete operations
- All methods delegate to CredentialManager for actual storage

* Wire IAM gRPC service to filer server

- Add CredentialManager field to FilerOption and FilerServer
- Import credential store implementations in filer command
- Initialize CredentialManager from credential.toml if available
- Register IAM gRPC service on filer gRPC server
- Enable credential management via gRPC alongside existing filer services

* Regenerate IAM protobuf with gRPC service methods

* fix: compilation error in DeleteUser

* fix: address code review comments for IAM migration

* feat: migrate policies to multi-file layout and fix identity duplicated content

* refactor: remove configuration.json and migrate Service Accounts to multi-file layout

* refactor: standardize Service Accounts as distinct store entities and fix Admin Server persistence

* config: set ServiceAccountsDirectory to /etc/iam/service_accounts

* Fix Chrome dialog auto-dismiss with Bootstrap modals

- Add modal-alerts.js library with Bootstrap modal replacements
- Replace all 15 confirm() calls with showConfirm/showDeleteConfirm
- Auto-override window.alert() for all alert() calls
- Fixes Chrome 132+ aggressively blocking native dialogs

* Upgrade Bootstrap from 5.3.2 to 5.3.8

* Fix syntax error in object_store_users.templ - remove duplicate closing braces

* create policy

* display errors

* migrate to multi-file policies

* address PR feedback: use showDeleteConfirm and showErrorMessage in policies.templ, refine migration check

* Update policies_templ.go

* add service account to iam grpc

* iam: fix potential path traversal in policy names by validating name pattern

* iam: add GetServiceAccountByAccessKey to CredentialStore interface

* iam: implement service account support for PostgresStore

Includes full CRUD operations and efficient lookup by access key.

* iam: implement GetServiceAccountByAccessKey for filer_etc, grpc, and memory stores

Provides efficient lookup of service accounts by access key where possible,
with linear scan fallbacks for file-based stores.

* iam: remove filer_multiple support

Deleted its implementation and references in imports, scaffold config,
and core interface constants. Redundant with filer_etc.

* clear comment

* dash: robustify service account construction

- Guard against nil sa.Credential when constructing responses
- Fix Expiration logic to only set if > 0, avoiding Unix epoch 1970
- Ensure consistency across Get, Create, and Update handlers

* credential/filer_etc: improve error propagation in configuration handlers

- Return error from loadServiceAccountsFromMultiFile to callers
- Ensure listEntries errors in SaveConfiguration (cleanup logic) are
  propagated unless they are "not found" failures.
- Fixes potential silent failures during IAM configuration sync.

* credential/filer_etc: add existence check to CreateServiceAccount

Ensures consistency with other stores by preventing accidental overwrite
of existing service accounts during creation.

* credential/memory: improve store robustness and Reset logic

- Enforce ID immutability in UpdateServiceAccount to prevent orphans
- Update Reset() to also clear the policies map, ensuring full state
  cleanup for tests.

* dash: improve service account robustness and policy docs

- Wrap parent user lookup errors to preserve context
- Strictly validate Status field in UpdateServiceAccount
- Add deprecation comments to legacy policy management methods

* credential/filer_etc: protect against path traversal in service accounts

Implemented ID validation (alphanumeric, underscores, hyphens) and applied
it to Get, Save, and Delete operations to ensure no directory traversal
via saId.json filenames.

* credential/postgres: improve robustness and cleanup comments

- Removed brainstorming comments in GetServiceAccountByAccessKey
- Added missing rows.Err() check during iteration
- Properly propagate Scan and Unmarshal errors instead of swallowing them

* admin: unify UI alerts and confirmations using Bootstrap modals

- Updated modal-alerts.js with improved automated alert type detection
- Replaced native alert() and confirm() with showAlert(), showConfirm(),
  and showDeleteConfirm() across various Templ components
- Improved UX for delete operations by providing better context and styling
- Ensured consistent error reporting across IAM and Maintenance views

* admin: additional UI consistency fixes for alerts and confirmations

- Replaced native alert() and confirm() with Bootstrap modals in:
  - EC volumes (repair flow)
  - Collection details (repair flow)
  - File browser (properties and delete)
  - Maintenance config schema (save and reset)
- Improved delete confirmation in file browser with item context
- Ensured consistent success/error/info styling for all feedbacks

* make

* iam: add GetServiceAccountByAccessKey RPC and update GetConfiguration

* iam: implement GetServiceAccountByAccessKey on server and client

* iam: centralize policy and service account validation

* iam: optimize MemoryStore service account lookups with indexing

* iam: fix postgres service_accounts table and optimize lookups

* admin: refactor modal alerts and clean up dashboard logic

* admin: fix EC shards table layout mismatch

* admin: URL-encode IAM path parameters for safety

* admin: implement pauseWorker logic in maintenance view

* iam: add rows.Err() check to postgres ListServiceAccounts

* iam: standardize ErrServiceAccountNotFound across credential stores

* iam: map ErrServiceAccountNotFound to codes.NotFound in DeleteServiceAccount

* iam: refine service account store logic, errors and schema

* iam: add validation to GetServiceAccountByAccessKey

* admin: refine modal titles and ensure URL safety

* admin: address bot review comments for alerts and async usage

* iam: fix syntax error by restoring missing function declaration

* [FilerEtcStore] improve error handling in CreateServiceAccount

Refine error handling to provide clearer messages when checking for
existing service accounts.

* [PostgresStore] add nil guards and validation to service account methods

Ensure input parameters are not nil and required IDs are present
to prevent runtime panics and ensure data integrity.

* [JS] add shared IAM utility script

Consolidate common IAM operations like deleteUser and deleteAccessKey
into a shared utility script for better maintainability.

* [View] include shared IAM utilities in layout

Include iam-utils.js in the main layout to make IAM functions
available across all administrative pages.

* [View] refactor IAM logic and restore async in EC Shards view

Remove redundant local IAM functions and ensure that delete
confirmation callbacks are properly marked as async.

* [View] consolidate IAM logic in Object Store Users view

Remove redundant local definitions of deleteUser and deleteAccessKey,
relying on the shared utilities instead.

* [View] update generated templ files for UI consistency

* credential/postgres: remove redundant name column from service_accounts table

The id is already used as the unique identifier and was being copied to the name column.
This removes the name column from the schema and updates the INSERT/UPDATE queries.

* credential/filer_etc: improve logging for policy migration failures

Added Errorf log if AtomicRenameEntry fails during migration to ensure visibility of common failure points.

* credential: allow uppercase characters in service account ID username

Updated ServiceAccountIdPattern to allow [A-Za-z0-9_-]+ for the username component,
matching the actual service account creation logic which uses the parent user name directly.

* Update object_store_users_templ.go

* admin: fix ec_shards pagination to handle numeric page arguments

Updated goToPage in cluster_ec_shards.templ to accept either an Event
or a numeric page argument. This prevents errors when goToPage(1)
is called directly. Corrected both the .templ source and generated Go code.

* credential/filer_etc: improve service account storage robustness

Added nil guard to saveServiceAccount, updated GetServiceAccount
to return ErrServiceAccountNotFound for empty data, and improved
deleteServiceAccount to handle response-level Filer errors.
2026-01-26 11:28:23 -08:00
Chris Lu
7e3bb4016e Fix syntax error in object_store_users.templ - remove duplicate closing braces 2026-01-25 23:18:40 -08:00
Chris Lu
74c7b10bc7 Fix Chrome dialog auto-dismiss with Bootstrap modals
- Add modal-alerts.js library with Bootstrap modal replacements
- Replace all 15 confirm() calls with showConfirm/showDeleteConfirm
- Auto-override window.alert() for all alert() calls
- Fixes Chrome 132+ aggressively blocking native dialogs
2026-01-25 23:09:14 -08:00
Chris Lu
ce23c4fca7 missing changes 2026-01-17 23:15:33 -08:00
Chris Lu
6bc5a64a98 Add access key status management to Admin UI (#8050)
* Add access key status management to Admin UI

- Add Status field to AccessKeyInfo struct
- Implement UpdateAccessKeyStatus API endpoint
- Add status dropdown in access keys modal
- Fix modal backdrop issue by using refreshAccessKeysList helper
- Status can be toggled between Active and Inactive

* Replace magic strings with constants for access key status

- Define AccessKeyStatusActive and AccessKeyStatusInactive constants in admin_data.go
- Define STATUS_ACTIVE and STATUS_INACTIVE constants in JavaScript
- Replace all hardcoded 'Active' and 'Inactive' strings with constants
- Update error messages to use constants for consistency

* Remove duplicate manageAccessKeys function definition

* Add security improvements to access key status management

- Add status validation in UpdateAccessKeyStatus to prevent invalid values
- Fix XSS vulnerability by replacing inline onchange with data attributes
- Add delegated event listener for status select changes
- Add URL encoding to API request path segments
2026-01-17 18:18:32 -08:00
Chris Lu
dbde8983a7 Fix bucket permission persistence in Admin UI (#8049)
Fix bucket permission persistence and security issues (#7226)

Security Fixes:
- Fix XSS vulnerability in showModal by using DOM methods instead of template strings for title
- Add escapeHtmlForAttribute helper to properly escape all HTML entities (&, <, >, ", ')
- Fix XSS in showSecretKey and showNewAccessKeyModal by using proper HTML escaping
- Fix XSS in createAccessKeysContent by replacing inline onclick with data attributes and event delegation

Code Cleanup:
- Remove debug label "(DEBUG)" from page header
- Remove debug console.log statements from buildBucketPermissionsNew
- Remove dead functions: addBucketPermissionRow, removeBucketPermissionRow, parseBucketPermissions, buildBucketPermissions

Validation Improvements:
- Add validation in handleUpdateUser to prevent empty permissions submission
- Update buildBucketPermissionsNew to return null when no buckets selected (instead of empty array)
- Add proper error messages for validation failures

UI Improvements:
- Enhanced access key management with proper modals and copy buttons
- Improved copy-to-clipboard functionality with fallbacks

Fixes #7226
2026-01-17 12:54:21 -08:00
Chris Lu
e67973dc53 Support Policy Attachment for Object Store Users (#7981)
* Implement Policy Attachment support for Object Store Users

- Added policy_names field to iam.proto and regenerated protos.
- Updated S3 API and IAM integration to support direct policy evaluation for users.
- Enhanced Admin UI to allow attaching policies to users via modals.
- Renamed 'policies' to 'policy_names' to clarify that it stores identifiers.
- Fixed syntax error in user_management.go.

* Fix policy dropdown not populating

The API returns {policies: [...]} but JavaScript was treating response as direct array.
Updated loadPolicies() to correctly access data.policies property.

* Add null safety checks for policy dropdowns

Added checks to prevent "undefined" errors when:
- Policy select elements don't exist
- Policy dropdowns haven't loaded yet
- User is being edited before policies are loaded

* Fix policy dropdown by using correct JSON field name

JSON response has lowercase 'name' field but JavaScript was accessing 'Name'.
Changed policy.Name to policy.name to match the IAMPolicy JSON structure.

* Fix policy names not being saved on user update

Changed condition from len(req.PolicyNames) > 0 to req.PolicyNames != nil
to ensure policy names are always updated when present in the request,
even if it's an empty array (to allow clearing policies).

* Add debug logging for policy names update flow

Added console.log in frontend and glog in backend to trace
policy_names data through the update process.

* Temporarily disable auto-reload for debugging

Commented out window.location.reload() so console logs are visible
when updating a user.

* Add detailed debug logging and alert for policy selection

Added console.log for each step and an alert to show policy_names value
to help diagnose why it's not being included in the request.

* Regenerate templ files for object_store_users

Ran templ generate to ensure _templ.go files are up to date with
the latest .templ changes including debug logging.

* Remove debug logging and restore normal functionality

Cleaned up temporary debug code (console.log and alert statements)
and re-enabled automatic page reload after user update.

* Add step-by-step alert debugging for policy update

Added 5 alert checkpoints to trace policy data through the update flow:
1. Check if policiesSelect element exists
2. Show selected policy values
3. Show userData.policy_names
4. Show full request body
5. Confirm server response

Temporarily disabled auto-reload to see alerts.

* Add version check alert on page load

Added alert on DOMContentLoaded to verify new JavaScript is being executed
and not cached by the browser.

* Compile templates using make

Ran make to compile all template files and install the weed binary.

* Add button click detection and make handleUpdateUser global

- Added inline alert on button click to verify click is detected
- Made handleUpdateUser a window-level function to ensure it's accessible
- Added alert at start of handleUpdateUser function

* Fix handleUpdateUser scope issue - remove duplicate definition

Removed duplicate function definition that was inside DOMContentLoaded.
Now handleUpdateUser is defined only once in global scope (line 383)
making it accessible when button onclick fires.

* Remove all duplicate handleUpdateUser definitions

Now handleUpdateUser is defined only once at the very top of the script
block (line 352), before DOMContentLoaded, ensuring it's available when
the button onclick fires.

* Add function existence check and error catching

Added alerts to check if handleUpdateUser is defined and wrapped
the function call in try-catch to capture any JavaScript errors.
Also added console.log statements to verify function definition.

* Simplify handleUpdateUser to non-async for testing

Removed async/await and added early return to test if function
can be called at all. This will help identify if async is causing
the issue.

* Add cache-control headers to prevent browser caching

Added no-cache headers to ShowObjectStoreUsers handler to prevent
aggressive browser caching of inline JavaScript in the HTML page.

* Fix syntax error - make handleUpdateUser async

Changed function back to async to fix 'await is only valid in async functions' error.
The cache-control headers are working - browser is now loading new code.

* Update version check to v3 to verify cache busting

Changed version alert to 'v3 - WITH EARLY RETURN' to confirm
the new code with early return statement is being loaded.

* Remove all debug code - clean implementation

Removed all alerts, console.logs, and test code.
Implemented clean policy update functionality with proper error handling.

* Add ETag header for cache-busting and update walkthrough

* Fix policy pre-selection in Edit User modal

- Updated admin.js editUser function to pre-select policies
- Root cause: duplicate editUser in admin.js overwrote inline version
- Added policy pre-selection logic to match inline template
- Verified working in browser: policies now pre-select correctly

* Fix policy persistence in handleUpdateUser

- Added policy_names field to userData payload in handleUpdateUser
- Policies were being lost because handleUpdateUser only sent email and actions
- Now collects selected policies from editPolicies dropdown
- Verified working: policies persist correctly across updates

* Fix XSS vulnerability in access keys display

- Escape HTML in access key display using escapeHtml utility
- Replace inline onclick handlers with data attributes
- Add event delegation for delete access key buttons
- Prevents script injection via malicious access key values

* Fix additional XSS vulnerabilities in user details display

- Escape HTML in actions badges (line 626)
- Escape HTML in policy_names badges (line 636)
- Prevents script injection via malicious action or policy names

* Fix XSS vulnerability in loadPolicies function

- Replace innerHTML string concatenation with DOM API
- Use createElement and textContent for safe policy name insertion
- Prevents script injection via malicious policy names
- Apply same pattern to both create and edit select elements

* Remove debug logging from UpdateObjectStoreUser

- Removed glog.V(0) debug statements
- Clean up temporary debugging code before production

* Remove duplicate handleUpdateUser function

- Removed inline handleUpdateUser that duplicated admin.js logic
- Removed debug console.log statement
- admin.js version is now the single source of truth
- Eliminates maintenance burden of keeping two versions in sync

* Refine user management and address code review feedback

- Preserve PolicyNames in UpdateUserPolicies
- Allow clearing actions in UpdateObjectStoreUser by checking for nil
- Remove version comment from object_store_users.templ
- Refactor loadPolicies for DRYness using cloneNode while keeping DOM API security

* IAM Authorization for Static Access Keys

* verified XSS Fixes in Templates

* fix div
2026-01-06 21:53:28 -08:00
Chris Lu
479e7bc38b go install github.com/a-h/templ/cmd/templ@latest 2025-10-20 22:22:12 -07:00
chrislu
e7dfc3552c admin ui adds object lock permissions 2025-07-13 20:29:25 -07:00
Chris Lu
687a6a6c1d Admin UI: Add policies (#6968)
* add policies to UI, accessing filer directly

* view, edit policies

* add back buttons for "users" page

* remove unused

* fix ui dark mode when modal is closed

* bucket view details button

* fix browser buttons

* filer action button works

* clean up masters page

* fix volume servers action buttons

* fix collections page action button

* fix properties page

* more obvious

* fix directory creation file mode

* Update file_browser_handlers.go

* directory permission
2025-07-12 01:13:11 -07:00
chrislu
3d4a9bdac0 upgrade templ version from v0.3.833 to v0.3.906
// templ: version: v0.3.833
// templ: version: v0.3.906

fix https://github.com/seaweedfs/seaweedfs/issues/6966#issuecomment-3063449163
2025-07-11 13:03:04 -07:00
chrislu
f47c4aef5a object store users 2025-07-02 00:00:23 -07:00
Chris Lu
1defee3d68 Add admin component (#6928)
* init version

* relocate

* add s3 bucket link

* refactor handlers into weed/admin folder

* fix login logout

* adding favicon

* remove fall back to http get topology

* grpc dial option, disk total capacity

* show filer count

* fix each volume disk usage

* add filers to dashboard

* adding hosts, volumes, collections

* refactor code and menu

* remove "refresh" button

* fix data for collections

* rename cluster hosts into volume servers

* add masters, filers

* reorder

* adding file browser

* create folder and upload files

* add filer version, created at time

* remove mock data

* remove fields

* fix submenu item highlighting

* fix bucket creation

* purge files

* delete multiple

* fix bucket creation

* remove region from buckets

* add object store with buckets and users

* rendering permission

* refactor

* get bucket objects and size

* link to file browser

* add file size and count for collections page

* paginate the volumes

* fix possible SSRF

https://github.com/seaweedfs/seaweedfs/pull/6928/checks?check_run_id=45108469801

* Update weed/command/admin.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update weed/command/admin.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix build

* import

* remove filer CLI option

* remove filer option

* remove CLI options

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-07-01 01:28:09 -07:00