Browse Source

debug: add comprehensive volume and container diagnostics

Added checks to diagnose why files aren't accessible:

1. Container status before restart
   - See if containers are still running or stopped
   - Check exit codes

2. Volume inspection
   - List all docker volumes
   - Inspect seaweedfs-volume-data volume
   - Check if volume data persisted

3. Access from inside container
   - Use curl from inside filer container
   - This bypasses host networking issues
   - Shows if files exist but aren't exposed

4. Direct filesystem check
   - Try to ls the directory from inside container
   - See if filer has filesystem access

This will definitively show:
- Did data persist through container restart?
- Are files there but not accessible via HTTP from host?
- Is the volume getting cleaned up somehow?
pull/7526/head
chrislu 1 week ago
parent
commit
2548ad91f7
  1. 19
      .github/workflows/spark-integration-tests.yml

19
.github/workflows/spark-integration-tests.yml

@ -132,6 +132,10 @@ jobs:
if: steps.test-run.outcome == 'failure'
working-directory: test/java/spark
run: |
echo "=== Checking containers status before restart ==="
docker compose ps -a
echo ""
echo "=== Restarting SeaweedFS services to access files ==="
docker compose up -d seaweedfs-master seaweedfs-volume seaweedfs-filer
@ -143,6 +147,11 @@ jobs:
fi
sleep 2
done
echo ""
echo "=== Volume status ==="
docker volume ls | grep spark || echo "No spark volumes found"
docker volume inspect test-java-spark_seaweedfs-volume-data 2>/dev/null || echo "Volume inspection failed"
- name: Download and examine Parquet files
if: steps.test-run.outcome == 'failure'
@ -155,11 +164,15 @@ jobs:
# First, check what's in the test-spark directory
echo "=== Checking test-spark directory structure ==="
echo -e "fs.ls /test-spark/\nexit" | docker compose exec -T seaweedfs-master weed shell
docker compose exec -T seaweedfs-filer sh -c "curl -s http://localhost:8888/test-spark/" || echo "Failed to list /test-spark/"
echo ""
echo "=== Checking employees directory content ==="
docker compose exec -T seaweedfs-filer sh -c "curl -s http://localhost:8888/test-spark/employees/" || echo "Failed to list /test-spark/employees/"
echo ""
echo "=== Checking employees directory ==="
echo -e "fs.ls /test-spark/employees/\nexit" | docker compose exec -T seaweedfs-master weed shell
echo "=== Checking volume data directory ==="
docker compose exec -T seaweedfs-filer sh -c "ls -la /test-spark/employees/ 2>&1" || echo "No direct filesystem access"
# List available files via HTTP
echo ""

Loading…
Cancel
Save