gsh-digital-services/seaweedFS
5
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

filer store: add foundationdb (#7178)

Browse Source 
* add foundationdb

* Update foundationdb_store.go

* fix

* apply the patch

* avoid panic on error

* address comments

* remove extra data

* address comments

* adds more debug messages

* fix range listing

* delete with prefix range; list with right start key

* fix docker files

* use the more idiomatic FoundationDB KeySelectors

* address comments

* proper errors

* fix API versions

* more efficient

* recursive deletion

* clean up

* clean up

* pagination, one transaction for deletion

* error checking

* Use fdb.Strinc() to compute the lexicographically next string and create a proper range

* fix docker

* Update README.md

* delete in batches

* delete in batches

* fix build

* add foundationdb build

* Updated FoundationDB Version

* Fixed glibc/musl Incompatibility (Alpine → Debian)

* Update container_foundationdb_version.yml

* build SeaweedFS

* build tag

* address comments

* separate transaction

* address comments

* fix build

* empty vs no data

* fixes

* add go test

* Install FoundationDB client libraries

* nil compare
This commit is contained in:
Chris Lu
2025-11-19 20:06:57 -08:00
committed by GitHub
parent 8be9e258fc
commit c6b6ea40e6
33 changed files with 5815 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

435
weed/filer/foundationdb/INSTALL.md Normal file
View File

@@ -0,0 +1,435 @@
# FoundationDB Filer Store Installation Guide
This guide covers the installation and setup of the FoundationDB filer store for SeaweedFS.
## Prerequisites
### FoundationDB Server
1. **Install FoundationDB Server**
**Ubuntu/Debian:**
```bash
# Add FoundationDB repository
curl -L https://github.com/apple/foundationdb/releases/download/7.4.5/foundationdb-clients_7.4.5-1_amd64.deb -o foundationdb-clients.deb
curl -L https://github.com/apple/foundationdb/releases/download/7.4.5/foundationdb-server_7.4.5-1_amd64.deb -o foundationdb-server.deb
sudo dpkg -i foundationdb-clients.deb foundationdb-server.deb
```
**CentOS/RHEL:**
```bash
# Install RPM packages
wget https://github.com/apple/foundationdb/releases/download/7.4.5/foundationdb-clients-7.4.5-1.el7.x86_64.rpm
wget https://github.com/apple/foundationdb/releases/download/7.4.5/foundationdb-server-7.4.5-1.el7.x86_64.rpm
sudo rpm -Uvh foundationdb-clients-7.4.5-1.el7.x86_64.rpm foundationdb-server-7.4.5-1.el7.x86_64.rpm
```
**macOS:**
```bash
# Using Homebrew (if available)
brew install foundationdb
# Or download from GitHub releases
# https://github.com/apple/foundationdb/releases
```
2. **Initialize FoundationDB Cluster**
**Single Node (Development):**
```bash
# Start FoundationDB service
sudo systemctl start foundationdb
sudo systemctl enable foundationdb
# Initialize database
fdbcli --exec 'configure new single ssd'
```
**Multi-Node Cluster (Production):**
```bash
# On each node, edit /etc/foundationdb/fdb.cluster
# Example: testing:testing@node1:4500,node2:4500,node3:4500
# On one node, initialize cluster
fdbcli --exec 'configure new double ssd'
```
3. **Verify Installation**
```bash
fdbcli --exec 'status'
```
### FoundationDB Client Libraries
The SeaweedFS FoundationDB integration requires the FoundationDB client libraries.
**Ubuntu/Debian:**
```bash
sudo apt-get install libfdb-dev
```
**CentOS/RHEL:**
```bash
sudo yum install foundationdb-devel
```
**macOS:**
```bash
# Client libraries are included with the server installation
export LIBRARY_PATH=/usr/local/lib
export CPATH=/usr/local/include
```
## Building SeaweedFS with FoundationDB Support
### Download FoundationDB Go Bindings
```bash
go mod init seaweedfs-foundationdb
go get github.com/apple/foundationdb/bindings/go/src/fdb
```
### Build SeaweedFS
```bash
# Clone SeaweedFS repository
git clone https://github.com/seaweedfs/seaweedfs.git
cd seaweedfs
# Build with FoundationDB support
go build -tags foundationdb -o weed
```
### Verify Build
```bash
./weed version
# Should show version information
./weed help
# Should list available commands
```
## Configuration
### Basic Configuration
Create or edit `filer.toml`:
```toml
[foundationdb]
enabled = true
cluster_file = "/etc/foundationdb/fdb.cluster"
api_version = 740
timeout = "5s"
max_retry_delay = "1s"
directory_prefix = "seaweedfs"
```
### Environment Variables
Alternative configuration via environment variables:
```bash
export WEED_FOUNDATIONDB_ENABLED=true
export WEED_FOUNDATIONDB_CLUSTER_FILE=/etc/foundationdb/fdb.cluster
export WEED_FOUNDATIONDB_API_VERSION=740
export WEED_FOUNDATIONDB_TIMEOUT=5s
export WEED_FOUNDATIONDB_MAX_RETRY_DELAY=1s
export WEED_FOUNDATIONDB_DIRECTORY_PREFIX=seaweedfs
```
### Advanced Configuration
For production deployments:
```toml
[foundationdb]
enabled = true
cluster_file = "/etc/foundationdb/fdb.cluster"
api_version = 740
timeout = "30s"
max_retry_delay = "5s"
directory_prefix = "seaweedfs_prod"
# Path-specific configuration for backups
[foundationdb.backup]
enabled = true
cluster_file = "/etc/foundationdb/fdb.cluster"
directory_prefix = "seaweedfs_backup"
location = "/backup"
timeout = "60s"
```
## Deployment
### Single Node Deployment
```bash
# Start SeaweedFS with FoundationDB filer
./weed server -filer \
-master.port=9333 \
-volume.port=8080 \
-filer.port=8888 \
-s3.port=8333
```
### Distributed Deployment
**Master Servers:**
```bash
# Node 1
./weed master -port=9333 -peers=master1:9333,master2:9333,master3:9333
# Node 2
./weed master -port=9333 -peers=master1:9333,master2:9333,master3:9333 -ip=master2
# Node 3
./weed master -port=9333 -peers=master1:9333,master2:9333,master3:9333 -ip=master3
```
**Filer Servers with FoundationDB:**
```bash
# Filer nodes
./weed filer -master=master1:9333,master2:9333,master3:9333 -port=8888
```
**Volume Servers:**
```bash
./weed volume -master=master1:9333,master2:9333,master3:9333 -port=8080
```
### Docker Deployment
**docker-compose.yml:**
```yaml
version: '3.9'
services:
foundationdb:
image: foundationdb/foundationdb:7.4.5
ports:
- "4500:4500"
volumes:
- fdb_data:/var/fdb/data
- fdb_config:/var/fdb/config
seaweedfs:
image: chrislusf/seaweedfs:latest
command: "server -filer -ip=seaweedfs"
ports:
- "9333:9333"
- "8888:8888"
- "8333:8333"
environment:
WEED_FOUNDATIONDB_ENABLED: "true"
WEED_FOUNDATIONDB_CLUSTER_FILE: "/var/fdb/config/fdb.cluster"
volumes:
- fdb_config:/var/fdb/config
depends_on:
- foundationdb
volumes:
fdb_data:
fdb_config:
```
### Kubernetes Deployment
**FoundationDB Operator:**
```bash
# Install FoundationDB operator
kubectl apply -f https://raw.githubusercontent.com/FoundationDB/fdb-kubernetes-operator/main/config/samples/deployment.yaml
```
**SeaweedFS with FoundationDB:**
```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: seaweedfs-filer
spec:
replicas: 3
selector:
matchLabels:
app: seaweedfs-filer
template:
metadata:
labels:
app: seaweedfs-filer
spec:
containers:
- name: seaweedfs
image: chrislusf/seaweedfs:latest
command: ["weed", "filer"]
env:
- name: WEED_FOUNDATIONDB_ENABLED
value: "true"
- name: WEED_FOUNDATIONDB_CLUSTER_FILE
value: "/var/fdb/config/cluster_file"
ports:
- containerPort: 8888
volumeMounts:
- name: fdb-config
mountPath: /var/fdb/config
volumes:
- name: fdb-config
configMap:
name: fdb-cluster-config
```
## Testing Installation
### Quick Test
```bash
# Start SeaweedFS with FoundationDB
./weed server -filer &
# Test file operations
echo "Hello FoundationDB" > test.txt
curl -F file=@test.txt "http://localhost:8888/test/"
curl "http://localhost:8888/test/test.txt"
# Test S3 API
curl -X PUT "http://localhost:8333/testbucket"
curl -T test.txt "http://localhost:8333/testbucket/test.txt"
```
### Integration Test Suite
```bash
# Run the provided test suite
cd test/foundationdb
make setup
make test
```
## Performance Tuning
### FoundationDB Tuning
```bash
# Configure for high performance
fdbcli --exec 'configure triple ssd'
fdbcli --exec 'configure storage_engine=ssd-redwood-1-experimental'
```
### SeaweedFS Configuration
```toml
[foundationdb]
enabled = true
cluster_file = "/etc/foundationdb/fdb.cluster"
timeout = "10s" # Longer timeout for large operations
max_retry_delay = "2s" # Adjust retry behavior
directory_prefix = "sw" # Shorter prefix for efficiency
```
### OS-Level Tuning
```bash
# Increase file descriptor limits
echo "* soft nofile 65536" >> /etc/security/limits.conf
echo "* hard nofile 65536" >> /etc/security/limits.conf
# Adjust network parameters
echo "net.core.rmem_max = 134217728" >> /etc/sysctl.conf
echo "net.core.wmem_max = 134217728" >> /etc/sysctl.conf
sysctl -p
```
## Monitoring and Maintenance
### Health Checks
```bash
# FoundationDB cluster health
fdbcli --exec 'status'
fdbcli --exec 'status details'
# SeaweedFS health
curl http://localhost:9333/cluster/status
curl http://localhost:8888/statistics/health
```
### Log Monitoring
**FoundationDB Logs:**
- `/var/log/foundationdb/` (default location)
- Monitor for errors, warnings, and performance issues
**SeaweedFS Logs:**
```bash
# Start with verbose logging
./weed -v=2 server -filer
```
### Backup and Recovery
**FoundationDB Backup:**
```bash
# Start backup
fdbbackup start -d file:///path/to/backup -t backup_tag
# Monitor backup
fdbbackup status -t backup_tag
# Restore from backup
fdbrestore start -r file:///path/to/backup -t backup_tag --wait
```
**SeaweedFS Metadata Backup:**
```bash
# Export filer metadata
./weed shell
> fs.meta.save /path/to/metadata/backup.gz
```
## Troubleshooting
### Common Issues
1. **Connection Refused**
- Check FoundationDB service status: `sudo systemctl status foundationdb`
- Verify cluster file: `cat /etc/foundationdb/fdb.cluster`
- Check network connectivity: `telnet localhost 4500`
2. **API Version Mismatch**
- Update API version in configuration
- Rebuild SeaweedFS with matching FDB client library
3. **Transaction Conflicts**
- Reduce transaction scope
- Implement appropriate retry logic
- Check for concurrent access patterns
4. **Performance Issues**
- Monitor cluster status: `fdbcli --exec 'status details'`
- Check data distribution: `fdbcli --exec 'status json'`
- Verify storage configuration
### Debug Mode
```bash
# Enable FoundationDB client tracing
export FDB_TRACE_ENABLE=1
export FDB_TRACE_PATH=/tmp/fdb_trace
# Start SeaweedFS with debug logging
./weed -v=3 server -filer
```
### Getting Help
1. **FoundationDB Documentation**: https://apple.github.io/foundationdb/
2. **SeaweedFS Community**: https://github.com/seaweedfs/seaweedfs/discussions
3. **Issue Reporting**: https://github.com/seaweedfs/seaweedfs/issues
For specific FoundationDB filer store issues, include:
- FoundationDB version and cluster configuration
- SeaweedFS version and build tags
- Configuration files (filer.toml)
- Error messages and logs
- Steps to reproduce the issue