Browse Source

fix: use container hostname for volume server to enable automatic volume creation

Root cause identified:
- Volume server was using -ip=127.0.0.1
- Master couldn't reach volume server at 127.0.0.1 from its container
- When Spark requested assignment, master tried to create volume via gRPC
- Master's gRPC call to 127.0.0.1:18080 failed (reached itself, not volume server)
- Result: 'No writable volumes' error

Solution:
- Change volume server to use -ip=seaweedfs-volume (container hostname)
- Master can now reach volume server at seaweedfs-volume:18080
- Automatic volume creation works as designed
- Kept -publicUrl=127.0.0.1:8080 for external clients (host network)

Workflow changes:
- Remove forced volume creation (curl POST to /vol/grow)
- Volumes will be created automatically on first write request
- Keep diagnostic output for troubleshooting
- Simplified startup verification

This matches how other SeaweedFS tests work with Docker networking.
pull/7526/head
chrislu 6 days ago
parent
commit
ce40e2fd58
  1. 50
      .github/workflows/spark-integration-tests.yml
  2. 2
      test/java/spark/docker-compose.yml

50
.github/workflows/spark-integration-tests.yml

@ -180,31 +180,10 @@ jobs:
curl -f http://localhost:9333/cluster/status || exit 1 curl -f http://localhost:9333/cluster/status || exit 1
curl -f http://localhost:8888/ || exit 1 curl -f http://localhost:8888/ || exit 1
# Check volume server registration and force volume creation
# Check volume server registration
echo "Checking volume server status..." echo "Checking volume server status..."
curl -s http://localhost:9333/dir/status | jq '.' || echo "jq not available" curl -s http://localhost:9333/dir/status | jq '.' || echo "jq not available"
echo "Triggering initial volume growth..."
curl -X POST "http://localhost:9333/vol/grow?replication=000&count=3" || echo "Volume growth request sent"
sleep 2
echo "Verifying volumes were created..."
for i in {1..10}; do
VOLUME_COUNT=$(curl -s http://localhost:9333/dir/status | jq -r '.Topology.DataCenters[0].Racks[0].DataNodes[0].Volumes // 0' 2>/dev/null || echo "0")
echo "Attempt $i/10: Volume count = $VOLUME_COUNT"
if [ "$VOLUME_COUNT" -gt 0 ] 2>/dev/null; then
echo "✓ Volume server has $VOLUME_COUNT volumes registered"
curl -s http://localhost:9333/dir/status | jq -r '.Topology.Layouts[]' || true
break
fi
if [ $i -eq 10 ]; then
echo "⚠️ Warning: No volumes created, tests may fail with 'No writable volumes'"
exit 1
fi
sleep 1
done
echo "✓ All SeaweedFS services are healthy and volumes are ready"
echo "✓ Volume server registered, volumes will be created on-demand during tests"
- name: Build Spark integration tests - name: Build Spark integration tests
working-directory: test/java/spark working-directory: test/java/spark
@ -378,31 +357,10 @@ jobs:
curl -f http://localhost:9333/cluster/status || exit 1 curl -f http://localhost:9333/cluster/status || exit 1
curl -f http://localhost:8888/ || exit 1 curl -f http://localhost:8888/ || exit 1
# Check volume server registration and force volume creation
# Check volume server registration
echo "Checking volume server status..." echo "Checking volume server status..."
curl -s http://localhost:9333/dir/status | jq '.' || echo "jq not available" curl -s http://localhost:9333/dir/status | jq '.' || echo "jq not available"
echo "Triggering initial volume growth..."
curl -X POST "http://localhost:9333/vol/grow?replication=000&count=3" || echo "Volume growth request sent"
sleep 2
echo "Verifying volumes were created..."
for i in {1..10}; do
VOLUME_COUNT=$(curl -s http://localhost:9333/dir/status | jq -r '.Topology.DataCenters[0].Racks[0].DataNodes[0].Volumes // 0' 2>/dev/null || echo "0")
echo "Attempt $i/10: Volume count = $VOLUME_COUNT"
if [ "$VOLUME_COUNT" -gt 0 ] 2>/dev/null; then
echo "✓ Volume server has $VOLUME_COUNT volumes registered"
curl -s http://localhost:9333/dir/status | jq -r '.Topology.Layouts[]' || true
break
fi
if [ $i -eq 10 ]; then
echo "⚠️ Warning: No volumes created, tests may fail with 'No writable volumes'"
exit 1
fi
sleep 1
done
echo "✓ All SeaweedFS services are healthy and volumes are ready"
echo "✓ Volume server registered, volumes will be created on-demand during tests"
- name: Build project - name: Build project
working-directory: test/java/spark working-directory: test/java/spark

2
test/java/spark/docker-compose.yml

@ -27,7 +27,7 @@ services:
ports: ports:
- "8080:8080" - "8080:8080"
- "18080:18080" - "18080:18080"
command: "volume -mserver=seaweedfs-master:9333 -ip=127.0.0.1 -ip.bind=0.0.0.0 -port=8080 -port.grpc=18080 -publicUrl=127.0.0.1:8080 -max=0 -dir=/data -preStopSeconds=1"
command: "volume -mserver=seaweedfs-master:9333 -ip=seaweedfs-volume -ip.bind=0.0.0.0 -port=8080 -port.grpc=18080 -publicUrl=127.0.0.1:8080 -max=0 -dir=/data -preStopSeconds=1"
volumes: volumes:
- seaweedfs-volume-data:/data - seaweedfs-volume-data:/data
depends_on: depends_on:

Loading…
Cancel
Save