Tree:
ae94e4d8f1
add-ec-vacuum
add-filer-iam-grpc
add-iam-grpc-management
add_fasthttp_client
add_remote_storage
adding-message-queue-integration-tests
adjust-fsck-cutoff-default
also-delete-parent-directory-if-empty
avoid_releasing_temp_file_on_write
changing-to-zap
chris/s3tables-shell-admin
collect-public-metrics
copilot/fix-helm-chart-installation
copilot/fix-s3-object-tagging-issue
copilot/make-renew-interval-configurable
copilot/make-renew-interval-configurable-again
copilot/sub-pr-7677
create-table-snapshot-api-design
data_query_pushdown
dependabot/maven/other/java/client/com.google.protobuf-protobuf-java-3.25.5
dependabot/maven/other/java/examples/org.apache.hadoop-hadoop-common-3.4.0
detect-and-plan-ec-tasks
do-not-retry-if-error-is-NotFound
ec-disk-type-support
enhance-erasure-coding
fasthttp
feature/mini-port-detection
feature/modernize-s3-tests
feature/s3-multi-cert-support
filer1_maintenance_branch
fix-GetObjectLockConfigurationHandler
fix-bucket-name-case-7910
fix-helm-fromtoml-compatibility
fix-mount-http-parallelism
fix-mount-read-throughput-7504
fix-pr-7909
fix-s3-configure-consistency
fix-s3-object-tagging-issue-7589
fix-sts-session-token-7941
fix-versioning-listing-only
fix/windows-test-file-cleanup
ftp
gh-pages
iam-multi-file-migration
iam-permissions-and-api
improve-fuse-mount
improve-fuse-mount2
logrus
master
message_send
mount2
mq-subscribe
mq2
nfs-cookie-prefix-list-fixes
optimize-delete-lookups
original_weed_mount
pr-7412
pr/7984
pr/8140
raft-dual-write
random_access_file
refactor-needle-read-operations
refactor-volume-write
remote_overlay
remove-implicit-directory-handling
revert-5134-patch-1
revert-5819-patch-1
revert-6434-bugfix-missing-s3-audit
s3-remote-cache-singleflight
s3-select
s3tables-by-claude
sub
tcp_read
test-reverting-lock-table
test_udp
testing
testing-sdx-generation
tikv
track-mount-e2e
upgrade-versions-to-4.00
volume_buffered_writes
worker-execute-ec-tasks
0.72
0.72.release
0.73
0.74
0.75
0.76
0.77
0.90
0.91
0.92
0.93
0.94
0.95
0.96
0.97
0.98
0.99
1.00
1.01
1.02
1.03
1.04
1.05
1.06
1.07
1.08
1.09
1.10
1.11
1.12
1.14
1.15
1.16
1.17
1.18
1.19
1.20
1.21
1.22
1.23
1.24
1.25
1.26
1.27
1.28
1.29
1.30
1.31
1.32
1.33
1.34
1.35
1.36
1.37
1.38
1.40
1.41
1.42
1.43
1.44
1.45
1.46
1.47
1.48
1.49
1.50
1.51
1.52
1.53
1.54
1.55
1.56
1.57
1.58
1.59
1.60
1.61
1.61RC
1.62
1.63
1.64
1.65
1.66
1.67
1.68
1.69
1.70
1.71
1.72
1.73
1.74
1.75
1.76
1.77
1.78
1.79
1.80
1.81
1.82
1.83
1.84
1.85
1.86
1.87
1.88
1.90
1.91
1.92
1.93
1.94
1.95
1.96
1.97
1.98
1.99
1;70
2.00
2.01
2.02
2.03
2.04
2.05
2.06
2.07
2.08
2.09
2.10
2.11
2.12
2.13
2.14
2.15
2.16
2.17
2.18
2.19
2.20
2.21
2.22
2.23
2.24
2.25
2.26
2.27
2.28
2.29
2.30
2.31
2.32
2.33
2.34
2.35
2.36
2.37
2.38
2.39
2.40
2.41
2.42
2.43
2.47
2.48
2.49
2.50
2.51
2.52
2.53
2.54
2.55
2.56
2.57
2.58
2.59
2.60
2.61
2.62
2.63
2.64
2.65
2.66
2.67
2.68
2.69
2.70
2.71
2.72
2.73
2.74
2.75
2.76
2.77
2.78
2.79
2.80
2.81
2.82
2.83
2.84
2.85
2.86
2.87
2.88
2.89
2.90
2.91
2.92
2.93
2.94
2.95
2.96
2.97
2.98
2.99
3.00
3.01
3.02
3.03
3.04
3.05
3.06
3.07
3.08
3.09
3.10
3.11
3.12
3.13
3.14
3.15
3.16
3.18
3.19
3.20
3.21
3.22
3.23
3.24
3.25
3.26
3.27
3.28
3.29
3.30
3.31
3.32
3.33
3.34
3.35
3.36
3.37
3.38
3.39
3.40
3.41
3.42
3.43
3.44
3.45
3.46
3.47
3.48
3.50
3.51
3.52
3.53
3.54
3.55
3.56
3.57
3.58
3.59
3.60
3.61
3.62
3.63
3.64
3.65
3.66
3.67
3.68
3.69
3.71
3.72
3.73
3.74
3.75
3.76
3.77
3.78
3.79
3.80
3.81
3.82
3.83
3.84
3.85
3.86
3.87
3.88
3.89
3.90
3.91
3.92
3.93
3.94
3.95
3.96
3.97
3.98
3.99
4.00
4.01
4.02
4.03
4.04
4.05
4.06
4.07
dev
helm-3.65.1
v0.69
v0.70beta
v3.33
${ noResults }
3 Commits (ae94e4d8f164ffd10b5be61fa28d4808dc3b7e8e)
| Author | SHA1 | Message | Date |
|---|---|---|---|
chrislu
|
5f8b632ff2 |
test: Run SeekToBeginningTest - BREAKTHROUGH: Metadata response advertising wrong hostname!
## Test Results ✅ SeekToBeginningTest.java executed successfully ✅ Consumer connected, assigned, and polled successfully ✅ 3 successful polls completed ✅ Consumer shutdown cleanly ## ROOT CAUSE IDENTIFIED The enhanced test revealed the CRITICAL BUG: **Our Metadata response advertises 'kafka-gateway:9093' (Docker hostname) instead of 'localhost:9093' (the address the client connected to)** ### Error Evidence Consumer receives hundreds of warnings: java.net.UnknownHostException: kafka-gateway at java.base/java.net.DefaultHostResolver.resolve() ### Why This Causes Schema Registry to Timeout 1. Client (Schema Registry) connects to kafka-gateway:9093 2. Gateway responds with Metadata 3. Metadata says broker is at 'kafka-gateway:9093' 4. Client tries to use that hostname 5. Name resolution works (Docker network) 6. BUT: Protocol response format or connectivity issue persists 7. Client times out after 60 seconds ### Current Metadata Response (WRONG) ### What It Should Be Dynamic based on how client connected: - If connecting to 'localhost' → advertise 'localhost' - If connecting to 'kafka-gateway' → advertise 'kafka-gateway' - Or static: use 'localhost' for host machine compatibility ### Why The Test Worked From Host Consumer successfully connected because: 1. Connected to localhost:9093 ✅ 2. Metadata said broker is kafka-gateway:9093 ❌ 3. Tried to resolve kafka-gateway from host ❌ 4. Failed resolution, but fallback polling worked anyway ✅ 5. Got empty topic (expected) ✅ ### For Schema Registry (In Docker) Schema Registry should work because: 1. Connects to kafka-gateway:9093 (both in Docker network) ✅ 2. Metadata says broker is kafka-gateway:9093 ✅ 3. Can resolve kafka-gateway (same Docker network) ✅ 4. Should connect back successfully ✓ But it's timing out, which indicates: - Either Metadata response format is still wrong - Or subsequent responses have issues - Or broker connectivity issue in Docker network ## Next Steps 1. Fix Metadata response to advertise correct hostname 2. Verify hostname matches client connection 3. Test again with Schema Registry 4. Debug if it still times out This is NOT a Kafka client bug. This is a **SeaweedFS Metadata advertisement bug**. |
4 months ago |
|
chrislu
|
7a509adc23 |
test: Enhanced SeekToBeginningTest with detailed request/response tracking
## What's New This enhanced Java diagnostic client adds detailed logging to understand exactly what the Kafka consumer is waiting for during seekToBeginning() + poll(): ### Features 1. **Detailed Exception Diagnosis** - Catches TimeoutException and reports what consumer is blocked on - Shows exception type and message - Suggests possible root causes 2. **Request/Response Tracking** - Shows when each operation completes or times out - Tracks timing for each poll() attempt - Reports records received vs expected 3. **Comprehensive Output** - Clear separation of steps (assign → seek → poll) - Summary statistics (successful/failed polls, total records) - Automated diagnosis of the issue 4. **Faster Feedback** - Reduced timeout from 30s to 15s per poll - Reduced default API timeout from 60s to 10s - Fails faster so we can iterate ### Expected Output **Success:** **Failure (what we're debugging):** ### How to Run ### Debugging Value This test will help us determine: 1. Is seekToBeginning() blocking? 2. Does poll() send ListOffsetsRequest? 3. Can consumer parse Metadata? 4. Are response messages malformed? 5. Is this a gateway bug or Kafka client issue? |
4 months ago |
|
chrislu
|
9a2d351a55 |
feat: Add standalone Java SeekToBeginning test to reproduce the issue
Created: - SeekToBeginningTest.java: Standalone Java test that reproduces the seekToBeginning() hang - Dockerfile.seektest: Docker setup for running the test - pom.xml: Maven build configuration - Updated docker-compose.yml to include seek-test service This test simulates what Schema Registry does: 1. Create KafkaConsumer connected to gateway 2. Assign to _schemas topic partition 0 3. Call seekToBeginning() 4. Poll for records Expected behavior: Should send ListOffsets and then Fetch Actual behavior: Blocks indefinitely after seekToBeginning() |
4 months ago |