gsh-digital-services/seaweedFS
5
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: auto-configure optimal volume size limit based on available disk space (#7833)

Browse Source 
* feat: auto-configure optimal volume size limit based on available disk space

- Add calculateOptimalVolumeSizeMB() function with OS-independent disk detection
- Reuses existing stats.NewDiskStatus() which works across Linux, macOS, Windows, BSD, Solaris
- Algorithm: available disk / 100, rounded up to nearest power of 2 (64MB, 128MB, 256MB, 512MB, 1024MB)
- Volume size capped to maximum of 1GB (1024MB) for better stability
- Minimum volume size is 64MB
- Uses efficient bits.Len() for power-of-2 rounding instead of floating-point operations
- Only auto-calculates volume size if user didn't specify a custom value via -master.volumeSizeLimitMB
- Respects user-specified values without override
- Master logs whether value was auto-calculated or user-specified
- Welcome message displays the configured volume size with correct format string ordering
- Removed unused autoVolumeSizeMB variable (logging handles source tracking)

Fixes: #0

* Refactor: Consolidate volume size constants and use robust flag detection for mini mode

This commit addresses all code review feedback on the auto-optimal volume size feature:

1. **Consolidate hardcoded defaults into package-level constants**
   - Moved minVolumeSizeMB=64 and maxVolumeSizeMB=1024 from local function-scope
     constants to package-level constants for consistency and maintainability
   - All three volume size constants (min, default, max) now defined in one place

2. **Implement robust flag detection using flag.Visit()**
   - Added isFlagPassed() helper function using flag.Visit() to check if a CLI
     flag was explicitly passed on the command line
   - Replaces the previous implementation that checked if current value equals
     default (which could incorrectly assume user intent if default was specified)
   - Now correctly detects user override regardless of the actual value

3. **Restructure power-of-2 rounding logic for clarity**
   - Changed from 'only round if above min threshold' to 'always round to power-of-2
     first, then apply min/max constraints'
   - More robust: works correctly even if min/max constants are adjusted in future
   - Clearer intent: all non-zero values go through consistent rounding logic

4. **Fix import ordering**
   - Added 'flag' import (aliased to fla9 package) to support isFlagPassed()
   - Added 'math/bits' import to support power-of-2 rounding

Benefits:
- Better code organization with all volume size limits in package constants
- Correct user override detection that doesn't rely on value equality checks
- More maintainable rounding logic that's easier to understand and modify
- Consistent with SeaweedFS conventions (uses fla9 package like other commands)

* fix: Address code review feedback for volume size calculation

This commit resolves three code review comments for better code quality and robustness:

1. **Handle comma-separated directories in -dir flag**
   - The -dir flag accepts comma-separated list of directories, but the volume size
     calculation was passing the entire string to util.ResolvePath()
   - Now splits on comma and uses the first directory for disk space calculation
   - Added explanatory comment about the multi-directory support
   - Ensures the optimal size calculation works correctly in all scenarios

2. **Change disk detection failure from verbose log to warning**
   - When disk status cannot be determined, the warning is now logged via
     glog.Warningf() instead of glog.V(1).Infof()
   - Makes the event visible in default logs without requiring verbose mode
   - Better alerting for operators about fallback to default values

3. **Avoid recalculating availableMB/100 and define bytesPerMB constant**
   - Added bytesPerMB = 1024*1024 constant for clarity and reusability
   - Replaced hardcoded (1024 * 1024) with bytesPerMB constant
   - Store availableMB/100 in initialOptimalMB variable to avoid recalculation
   - Log message now references initialOptimalMB instead of recalculating
   - Improves maintainability and reduces redundant computation

All three changes maintain the same logic while improving code quality and
robustness as requested by the reviewer.

* fix: Address rounding logic, logging clarity, and disk capacity measurement issues

This commit resolves three additional code review comments to improve robustness
and clarity of the volume size calculation:

1. **Fix power-of-2 rounding logic for edge cases**
   - The previous condition 'if optimalMB > 0' created a bug: when optimalMB=1,
     bits.Len(0)=0, resulting in 1<<0=1, which is below minimum (64MB)
   - Changed to explicitly handle zero case first: 'if optimalMB == 0'
   - Separate zero-handling from power-of-2 rounding ensures correct behavior:
     * optimalMB=0 → set to minVolumeSizeMB (64)
     * optimalMB>=1 → apply power-of-2 rounding
   - Then apply min/max constraints unconditionally
   - More explicit and easier to reason about correctness

2. **Use total disk capacity instead of free space for stable configuration**
   - Changed from diskStatus.Free (available space) to diskStatus.All (total capacity)
   - Free space varies based on current disk usage at startup time
   - This caused inconsistent volume sizes: same disk could get different sizes
     depending on how full it is when the service starts
   - Using total capacity ensures predictable, stable configuration across restarts
   - Better aligns with the intended behavior of sizing based on disk capacity
   - Added explanatory comments about why total capacity is more appropriate

3. **Improve log message clarity and accuracy**
   - Updated message to clearly show:
     * 'total disk capacity' instead of vague 'available disk'
     * 'capacity/100 before rounding' to match actual calculation
     * 'clamped to [min,max]' instead of 'capped to max' to show both bounds
     * Includes min and max values in log for context
   - More accurate and helpful for operators troubleshooting volume sizing

These changes ensure the volume size calculation is both correct and predictable.

* feat: Save mini configuration to file for persistence and documentation

This commit adds persistent configuration storage for the 'weed mini' command,
saving all non-default parameters to a JSON configuration file for:

1. **Configuration Documentation**
   - All parameters actually passed on the command line are saved
   - Provides a clear record of the running configuration
   - Useful for auditing and understanding how the system is configured

2. **Persistence of Auto-Calculated Values**
   - The auto-calculated optimal volume size (master.volumeSizeLimitMB) is saved
     with a note indicating it was auto-calculated
   - On restart, if the auto-calculated value exists, it won't be recalculated
   - Users can delete the auto-calculated entry to force recalculation on next startup
   - Provides stable, predictable configuration across restarts

3. **Configuration File Location**
   - Saved to: <data-folder>/.seaweedfs/mini.config.json
   - Uses the first directory from comma-separated -dir list
   - Directory is created automatically if it doesn't exist
   - JSON format for easy parsing and manual editing

4. **Implementation Details**
   - Uses flag.Visit() to collect only explicitly passed flags
   - Distinguishes between user-specified and auto-calculated values
   - Includes helpful notes in the JSON file
   - Graceful handling of save errors (logs warnings, doesn't fail startup)

The configuration file includes all parameters such as:
- IP and port settings (master, filer, volume, admin)
- Data directories and metadata folders
- Replication and collection settings
- S3 and IAM configurations
- Performance tuning parameters (concurrency limits, timeouts, etc.)
- Auto-calculated volume size (if applicable)

Example mini.config.json output:
{
  "debug": "true",
  "dir": "/data/seaweedfs",
  "master.port": "9333",
  "filer.port": "8888",
  "volume.port": "9340",
  "master.volumeSizeLimitMB.auto": "256",
  "_note_auto_calculated": "This value was auto-calculated. Remove it to recalculate on next startup."
}

This allows operators to:
- Review what configuration was active
- Replicate the configuration on other systems
- Understand the startup behavior
- Control when auto-calculation occurs

* refactor: Change configuration file format to match command-line options format

Update the saved configuration format from JSON to shell-compatible options format
that matches how options are expected to be passed on the command line.

Configuration file: .seaweedfs/mini.options

Format: Each line contains a command-line option in the format -name=value

Benefits:
- Format is compatible with shell scripts and can be sourced
- Can be easily converted to command-line options
- Human-readable and editable
- Values with spaces are properly quoted
- Includes helpful comments explaining auto-calculated values
- Directly usable with weed mini command

The file can be used in multiple ways:
1. Extract options: cat .seaweedfs/mini.options | grep -v '^#' | tr '\n' ' '
2. Inline in command: weed mini \$(cat .seaweedfs/mini.options | grep -v '^#')
3. Manual review: cat .seaweedfs/mini.options

* refactor: Save mini.options directly to -dir folder

* docs: Update PR description with accurate algorithm and examples

Update the function documentation comments to accurately reflect the implemented
algorithm and provide real-world examples with actual calculated outputs.

Changes:
- Clarify that algorithm uses total disk capacity (not free space)
- Document exact calculation: capacity/100, round to power of 2, clamp to [64,1024]
- Add realistic examples showing input disk sizes and resulting volume sizes:
  * 10GB disk → 64MB (minimum)
  * 100GB disk → 64MB (minimum)
  * 1TB disk → 64MB (minimum)
  * 6.4TB disk → 64MB
  * 12.8TB disk → 128MB
  * 100TB disk → 1024MB (maximum)
  * 1PB disk → 1024MB (maximum)
- Include note that values are rounded to next power of 2 and capped at 1GB

This helps users understand the volume size calculation and predict what size
will be set for their specific disk configurations.

* feat: integrate configuration file loading into mini startup

- Load mini.options file at startup if it exists
- Apply loaded configuration options before normal initialization
- CLI flags override file-based configuration
- Exclude 'dir' option from being saved (environment-specific)
- Configuration file format: option=value without leading dashes
- Auto-calculated volume size persists with recalculation marker
This commit is contained in:
Chris Lu
2025-12-21 12:47:27 -08:00
committed by GitHub
parent 3613279f25
commit 31cb28d9d3
1 changed files with 216 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files

220
weed/command/mini.go
View File

@@ -3,6 +3,7 @@ package command
import (
"context"
"fmt"
"math/bits"
"net"
"net/http"
"os"
@@ -17,6 +18,7 @@ import (
"github.com/seaweedfs/seaweedfs/weed/security"
stats_collect "github.com/seaweedfs/seaweedfs/weed/stats"
"github.com/seaweedfs/seaweedfs/weed/util"
flag "github.com/seaweedfs/seaweedfs/weed/util/fla9"
"github.com/seaweedfs/seaweedfs/weed/util/grace"
"github.com/seaweedfs/seaweedfs/weed/worker"
"github.com/seaweedfs/seaweedfs/weed/worker/types"
@@ -36,8 +38,12 @@ type MiniOptions struct {
}
const (
miniVolumeMaxDataVolumeCounts = "0" // auto-configured based on free disk space
miniVolumeMinFreeSpace = "1" // 1% minimum free space
bytesPerMB = 1024 * 1024 // Bytes per MB
miniVolumeMaxDataVolumeCounts = "0" // auto-configured based on free disk space
miniVolumeMinFreeSpace = "1" // 1% minimum free space
minVolumeSizeMB = 64 // Minimum volume size in MB
defaultMiniVolumeSizeMB = 128 // Default volume size for mini mode
maxVolumeSizeMB = 1024 // Maximum volume size in MB (1GB)
)
var (
@@ -125,7 +131,7 @@ func initMiniMasterFlags() {
miniMasterOptions.portGrpc = cmdMini.Flag.Int("master.port.grpc", 0, "master server grpc listen port")
miniMasterOptions.metaFolder = cmdMini.Flag.String("master.dir", "", "data directory to store meta data, default to same as -dir specified")
miniMasterOptions.peers = cmdMini.Flag.String("master.peers", "", "all master nodes in comma separated ip:masterPort list (default: none for single master)")
miniMasterOptions.volumeSizeLimitMB = cmdMini.Flag.Uint("master.volumeSizeLimitMB", 128, "Master stops directing writes to oversized volumes (default: 128MB for mini)")
miniMasterOptions.volumeSizeLimitMB = cmdMini.Flag.Uint("master.volumeSizeLimitMB", defaultMiniVolumeSizeMB, "Master stops directing writes to oversized volumes (default: 128MB for mini)")
miniMasterOptions.volumePreallocate = cmdMini.Flag.Bool("master.volumePreallocate", false, "Preallocate disk space for volumes.")
miniMasterOptions.maxParallelVacuumPerServer = cmdMini.Flag.Int("master.maxParallelVacuumPerServer", 1, "maximum number of volumes to vacuum in parallel on one volume server")
miniMasterOptions.defaultReplication = cmdMini.Flag.String("master.defaultReplication", "", "Default replication type if not specified.")
@@ -256,8 +262,196 @@ func init() {
initMiniAdminFlags()
}
// calculateOptimalVolumeSizeMB calculates optimal volume size based on total disk capacity.
//
// Algorithm:
// 1. Read total disk capacity using the OS-independent stats.NewDiskStatus()
// 2. Divide total disk capacity by 100 to estimate optimal volume size
// 3. Round up to nearest power of 2 (64MB, 128MB, 256MB, 512MB, 1024MB, etc.)
// 4. Clamp the result to range [64MB, 1024MB]
//
// Examples (values are rounded to next power of 2 and capped at 1GB):
// - 10GB disk → 10 / 100 = 0.1MB → rounds to 64MB (minimum)
// - 100GB disk → 100 / 100 = 1MB → rounds to 1MB, clamped to 64MB (minimum)
// - 500GB disk → 500 / 100 = 5MB → rounds to 8MB, clamped to 64MB (minimum)
// - 1TB disk → 1000 / 100 = 10MB → rounds to 16MB, clamped to 64MB (minimum)
// - 6.4TB disk → 6400 / 100 = 64MB → rounds to 64MB
// - 12.8TB disk → 12800 / 100 = 128MB → rounds to 128MB
// - 100TB disk → 100000 / 100 = 1000MB → rounds to 1024MB (maximum)
// - 1PB disk → 1000000 / 100 = 10000MB → capped at 1024MB (maximum)
func calculateOptimalVolumeSizeMB(dataFolder string) uint {
// Get disk status for the data folder using OS-independent function
diskStatus := stats_collect.NewDiskStatus(dataFolder)
if diskStatus == nil || diskStatus.All == 0 {
glog.Warningf("Could not determine disk size, using default %dMB", defaultMiniVolumeSizeMB)
return defaultMiniVolumeSizeMB
}
// Calculate optimal size: total disk capacity / 100 for stability
// Using total capacity (All) instead of free space ensures consistent volume size
// regardless of current disk usage. diskStatus.All is in bytes, convert to MB
totalCapacityMB := diskStatus.All / bytesPerMB
initialOptimalMB := uint(totalCapacityMB / 100)
optimalMB := initialOptimalMB
// Round up to nearest power of 2: 64MB, 128MB, 256MB, 512MB, etc.
// Minimum is 64MB, maximum is 1024MB (1GB)
if optimalMB == 0 {
// If the computed optimal size is 0, start from the minimum volume size
optimalMB = minVolumeSizeMB
} else {
// Round up to the nearest power of 2
optimalMB = 1 << bits.Len(optimalMB-1)
}
// Apply the minimum and maximum constraints
if optimalMB < minVolumeSizeMB {
optimalMB = minVolumeSizeMB
} else if optimalMB > maxVolumeSizeMB {
optimalMB = maxVolumeSizeMB
}
glog.Infof("Optimal volume size: %dMB (total disk capacity: %dMB, capacity/100 before rounding: %dMB, rounded to nearest power of 2, clamped to [%d,%d]MB)",
optimalMB, totalCapacityMB, initialOptimalMB, minVolumeSizeMB, maxVolumeSizeMB)
return optimalMB
}
// isFlagPassed checks if a specific flag was passed on the command line
func isFlagPassed(name string) bool {
found := false
cmdMini.Flag.Visit(func(f *flag.Flag) {
if f.Name == name {
found = true
}
})
return found
}
// loadMiniConfigurationFile reads the mini.options file and returns parsed options
// File format: one option per line, without leading dash (e.g., "ip=127.0.0.1")
func loadMiniConfigurationFile(dataFolder string) (map[string]string, error) {
configFile := filepath.Join(util.ResolvePath(util.StringSplit(dataFolder, ",")[0]), "mini.options")
options := make(map[string]string)
// Check if file exists
data, err := os.ReadFile(configFile)
if err != nil {
if os.IsNotExist(err) {
// File doesn't exist - this is OK, return empty options
return options, nil
}
glog.Warningf("Failed to read configuration file %s: %v", configFile, err)
return options, err
}
// Parse the file line by line
lines := strings.Split(string(data), "\n")
for _, line := range lines {
line = strings.TrimSpace(line)
// Skip empty lines and comments
if len(line) == 0 || strings.HasPrefix(line, "#") {
continue
}
// Remove leading dash if present
if strings.HasPrefix(line, "-") {
line = line[1:]
}
// Parse key=value
parts := strings.SplitN(line, "=", 2)
if len(parts) == 2 {
key := strings.TrimSpace(parts[0])
value := strings.TrimSpace(parts[1])
// Remove quotes if present
if (strings.HasPrefix(value, "\"") && strings.HasSuffix(value, "\"")) ||
(strings.HasPrefix(value, "'") && strings.HasSuffix(value, "'")) {
value = value[1 : len(value)-1]
}
options[key] = value
}
}
glog.Infof("Loaded %d options from configuration file %s", len(options), configFile)
return options, nil
}
// applyConfigFileOptions sets command-line flags from loaded configuration file
func applyConfigFileOptions(options map[string]string) {
for key, value := range options {
// Set the flag value if it hasn't been explicitly set on command line
flag := cmdMini.Flag.Lookup(key)
if flag != nil {
// Only set if not already set (by command line)
if flag.Value.String() == flag.DefValue {
flag.Value.Set(value)
glog.V(2).Infof("Applied config file option: %s=%s", key, value)
}
}
}
}
// saveMiniConfiguration saves the current mini configuration to a file
// The file format uses option=value format without leading dashes
func saveMiniConfiguration(dataFolder string) error {
configDir := util.ResolvePath(util.StringSplit(dataFolder, ",")[0])
if err := os.MkdirAll(configDir, 0755); err != nil {
glog.Warningf("Failed to create config directory %s: %v", configDir, err)
return err
}
configFile := filepath.Join(configDir, "mini.options")
var sb strings.Builder
sb.WriteString("#!/bin/bash\n")
sb.WriteString("# Mini server configuration\n")
sb.WriteString("# Format: option=value (no leading dash)\n")
sb.WriteString("# This file is loaded on startup if it exists\n\n")
// Collect all flags that were explicitly passed (except "dir")
cmdMini.Flag.Visit(func(f *flag.Flag) {
// Skip the "dir" option - it's environment-specific
if f.Name == "dir" {
return
}
value := f.Value.String()
// Quote the value if it contains spaces
if strings.Contains(value, " ") {
sb.WriteString(fmt.Sprintf("%s=\"%s\"\n", f.Name, value))
} else {
sb.WriteString(fmt.Sprintf("%s=%s\n", f.Name, value))
}
})
// Add auto-calculated volume size if it was computed
if !isFlagPassed("master.volumeSizeLimitMB") && miniMasterOptions.volumeSizeLimitMB != nil {
sb.WriteString(fmt.Sprintf("\n# Auto-calculated volume size based on total disk capacity\n"))
sb.WriteString(fmt.Sprintf("# Delete this line to force recalculation on next startup\n"))
sb.WriteString(fmt.Sprintf("master.volumeSizeLimitMB=%d\n", *miniMasterOptions.volumeSizeLimitMB))
}
if err := os.WriteFile(configFile, []byte(sb.String()), 0644); err != nil {
glog.Warningf("Failed to save configuration to %s: %v", configFile, err)
return err
}
glog.Infof("Mini configuration saved to %s", configFile)
return nil
}
func runMini(cmd *Command, args []string) bool {
// Load configuration from file if it exists
configOptions, err := loadMiniConfigurationFile(*miniDataFolders)
if err != nil {
glog.Warningf("Error loading configuration file: %v", err)
}
// Apply loaded options to flags (CLI flags will override these)
applyConfigFileOptions(configOptions)
if *miniOptions.debug {
grace.StartDebugServer(*miniOptions.debugPort)
}
@@ -325,6 +519,20 @@ func runMini(cmd *Command, args []string) bool {
}
miniFilerOptions.defaultLevelDbDirectory = miniMasterOptions.metaFolder
// Calculate and set optimal volume size limit based on available disk space
// Only auto-calculate if user didn't explicitly specify a value via -master.volumeSizeLimitMB
if !isFlagPassed("master.volumeSizeLimitMB") {
// User didn't override, use auto-calculated value
// The -dir flag can accept comma-separated directories; use the first one for disk space calculation
resolvedDataFolder := util.ResolvePath(util.StringSplit(*miniDataFolders, ",")[0])
optimalVolumeSizeMB := calculateOptimalVolumeSizeMB(resolvedDataFolder)
miniMasterOptions.volumeSizeLimitMB = &optimalVolumeSizeMB
glog.Infof("Mini started with auto-calculated optimal volume size limit: %dMB", optimalVolumeSizeMB)
} else {
// User specified a custom value
glog.Infof("Mini started with user-specified volume size limit: %dMB", *miniMasterOptions.volumeSizeLimitMB)
}
miniWhiteList := util.StringSplit(*miniWhiteListOption, ",")
// Start all services with proper dependency coordination
@@ -338,6 +546,9 @@ func runMini(cmd *Command, args []string) bool {
// Print welcome message after all services are running
printWelcomeMessage()
// Save configuration to file for persistence and documentation
saveMiniConfiguration(*miniDataFolders)
select {}
}
@@ -633,7 +844,7 @@ const welcomeMessageTemplate = `
Volume Server: http://%s:%d
Optimized Settings:
• Volume size limit: 128MB
• Volume size limit: %dMB
• Volume max: auto (based on free disk space)
• Pre-stop seconds: 1 (faster shutdown)
• Master peers: none (single master mode)
@@ -673,6 +884,7 @@ func printWelcomeMessage() {
*miniIp, *miniWebDavOptions.port,
*miniIp, *miniAdminOptions.port,
*miniIp, *miniOptions.v.port,
*miniMasterOptions.volumeSizeLimitMB,
*miniDataFolders,
)