551a31e156
* Implement IAM propagation to S3 servers - Add PropagatingCredentialStore to propagate IAM changes to S3 servers via gRPC - Add Policy management RPCs to S3 proto and S3ApiServer - Update CredentialManager to use PropagatingCredentialStore when MasterClient is available - Wire FilerServer to enable propagation * Implement parallel IAM propagation and fix S3 cluster registration - Parallelized IAM change propagation with 10s timeout. - Refined context usage in PropagatingCredentialStore. - Added S3Type support to cluster node management. - Enabled S3 servers to register with gRPC address to the master. - Ensured IAM configuration reload after policy updates via gRPC. * Optimize IAM propagation with direct in-memory cache updates * Secure IAM propagation: Use metadata to skip persistence only on propagation * pb: refactor IAM and S3 services for unidirectional IAM propagation - Move SeaweedS3IamCache service from iam.proto to s3.proto. - Remove legacy IAM management RPCs and empty SeaweedS3 service from s3.proto. - Enforce that S3 servers only use the synchronization interface. * pb: regenerate Go code for IAM and S3 services Updated generated code following the proto refactoring of IAM synchronization services. * s3api: implement read-only mode for Embedded IAM API - Add readOnly flag to EmbeddedIamApi to reject write operations via HTTP. - Enable read-only mode by default in S3ApiServer. - Handle AccessDenied error in writeIamErrorResponse. - Embed SeaweedS3IamCacheServer in S3ApiServer. * credential: refactor PropagatingCredentialStore for unidirectional IAM flow - Update to use s3_pb.SeaweedS3IamCacheClient for propagation to S3 servers. - Propagate full Identity object via PutIdentity for consistency. - Remove redundant propagation of specific user/account/policy management RPCs. - Add timeout context for propagation calls. * s3api: implement SeaweedS3IamCacheServer for unidirectional sync - Update S3ApiServer to implement the cache synchronization gRPC interface. - Methods (PutIdentity, RemoveIdentity, etc.) now perform direct in-memory cache updates. - Register SeaweedS3IamCacheServer in command/s3.go. - Remove registration for the legacy and now empty SeaweedS3 service. * s3api: update tests for read-only IAM and propagation - Added TestEmbeddedIamReadOnly to verify rejection of write operations in read-only mode. - Update test setup to pass readOnly=false to NewEmbeddedIamApi in routing tests. - Updated EmbeddedIamApiForTest helper with read-only checks matching production behavior. * s3api: add back temporary debug logs for IAM updates Log IAM updates received via: - gRPC propagation (PutIdentity, PutPolicy, etc.) - Metadata configuration reloads (LoadS3ApiConfigurationFromCredentialManager) - Core identity management (UpsertIdentity, RemoveIdentity) * IAM: finalize propagation fix with reduced logging and clarified architecture * Allow configuring IAM read-only mode for S3 server integration tests * s3api: add defensive validation to UpsertIdentity * s3api: fix log message to reference correct IAM read-only flag * test/s3/iam: ensure WaitForS3Service checks for IAM write permissions * test: enable writable IAM in Makefile for integration tests * IAM: add GetPolicy/ListPolicies RPCs to s3.proto * S3: add GetBucketPolicy and ListBucketPolicies helpers * S3: support storing generic IAM policies in IdentityAccessManagement * S3: implement IAM policy RPCs using IdentityAccessManagement * IAM: fix stale user identity on rename propagation
91 lines
2.5 KiB
Bash
Executable File
91 lines
2.5 KiB
Bash
Executable File
#!/bin/bash
|
|
# Test runner for S3 policy variables integration tests
|
|
# This script starts a SeaweedFS server with the required IAM configuration
|
|
# and runs the integration tests.
|
|
|
|
set -e
|
|
|
|
# Colors for output
|
|
RED='\033[0;31m'
|
|
GREEN='\033[0;32m'
|
|
YELLOW='\033[1;33m'
|
|
NC='\033[0m' # No Color
|
|
|
|
echo -e "${GREEN}=== S3 Policy Variables Integration Test Runner ===${NC}"
|
|
|
|
# Get the directory of this script
|
|
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|
PROJECT_ROOT="$(cd "$SCRIPT_DIR/../../.." && pwd)"
|
|
|
|
# Always build to ensure latest changes are tested
|
|
echo -e "${YELLOW}Building weed binary...${NC}"
|
|
cd "$PROJECT_ROOT/weed" && go install
|
|
if ! command -v weed &> /dev/null; then
|
|
echo -e "${RED}Failed to build weed binary${NC}"
|
|
exit 1
|
|
fi
|
|
|
|
# Kill any existing weed server on port 8333
|
|
echo "Checking for existing weed server..."
|
|
if lsof -Pi :8333 -sTCP:LISTEN -t >/dev/null 2>&1 ; then
|
|
echo -e "${YELLOW}Killing existing weed server on port 8333...${NC}"
|
|
kill $(lsof -t -i:8333) 2>/dev/null || true
|
|
sleep 2
|
|
fi
|
|
|
|
# Start weed server with IAM configuration
|
|
echo -e "${GREEN}Starting weed server with IAM configuration...${NC}"
|
|
weed server \
|
|
-s3 \
|
|
-s3.port=8333 \
|
|
-s3.iam.config="$SCRIPT_DIR/test_iam_config.json" \
|
|
-filer \
|
|
-volume.max=0 \
|
|
-master.volumeSizeLimitMB=100 \
|
|
-s3.allowDeleteBucketNotEmpty=true \
|
|
-s3.iam.readOnly=false \
|
|
> /tmp/weed_test_server.log 2>&1 &
|
|
|
|
SERVER_PID=$!
|
|
echo "Server started with PID: $SERVER_PID"
|
|
|
|
# Wait for server to be ready
|
|
echo "Waiting for server to be ready..."
|
|
MAX_WAIT=30
|
|
COUNTER=0
|
|
while ! curl -s http://localhost:8333/status > /dev/null 2>&1; do
|
|
sleep 1
|
|
COUNTER=$((COUNTER + 1))
|
|
if [ $COUNTER -ge $MAX_WAIT ]; then
|
|
echo -e "${RED}Server failed to start within ${MAX_WAIT} seconds${NC}"
|
|
echo "Server log:"
|
|
cat /tmp/weed_test_server.log
|
|
kill $SERVER_PID 2>/dev/null || true
|
|
exit 1
|
|
fi
|
|
done
|
|
|
|
echo -e "${GREEN}Server is ready!${NC}"
|
|
|
|
# Run the tests
|
|
echo -e "${GREEN}Running integration tests...${NC}"
|
|
cd "$SCRIPT_DIR"
|
|
|
|
# Trap to ensure server is killed on exit
|
|
trap "echo -e '${YELLOW}Shutting down server...${NC}'; kill $SERVER_PID 2>/dev/null || true" EXIT
|
|
|
|
# Run the tests
|
|
go test -v -run TestS3PolicyVariables .
|
|
|
|
TEST_RESULT=$?
|
|
|
|
if [ $TEST_RESULT -eq 0 ]; then
|
|
echo -e "${GREEN}=== All tests passed! ===${NC}"
|
|
else
|
|
echo -e "${RED}=== Tests failed ===${NC}"
|
|
echo "Server log (last 50 lines):"
|
|
tail -50 /tmp/weed_test_server.log
|
|
fi
|
|
|
|
exit $TEST_RESULT
|