Tree:
b68b9c6dd6
add-ec-vacuum
add-weed-mini-command
add_fasthttp_client
add_remote_storage
adding-message-queue-integration-tests
adjust-fsck-cutoff-default
also-delete-parent-directory-if-empty
avoid_releasing_temp_file_on_write
changing-to-zap
collect-public-metrics
copilot/fix-helm-chart-installation
copilot/fix-s3-object-tagging-issue
copilot/make-renew-interval-configurable
copilot/make-renew-interval-configurable-again
copilot/sub-pr-7677
create-table-snapshot-api-design
data_query_pushdown
dependabot/maven/other/java/client/com.google.protobuf-protobuf-java-3.25.5
dependabot/maven/other/java/examples/org.apache.hadoop-hadoop-common-3.4.0
detect-and-plan-ec-tasks
do-not-retry-if-error-is-NotFound
ec-disk-type-support
enhance-erasure-coding
fasthttp
filer1_maintenance_branch
fix-GetObjectLockConfigurationHandler
fix-mount-http-parallelism
fix-mount-read-throughput-7504
fix-s3-object-tagging-issue-7589
fix-versioning-listing-only
ftp
gh-pages
improve-fuse-mount
improve-fuse-mount2
logrus
master
message_send
mount2
mq-subscribe
mq2
nfs-cookie-prefix-list-fixes
optimize-delete-lookups
original_weed_mount
pr-7412
raft-dual-write
random_access_file
refactor-needle-read-operations
refactor-volume-write
remote_overlay
remove-implicit-directory-handling
revert-5134-patch-1
revert-5819-patch-1
revert-6434-bugfix-missing-s3-audit
s3-remote-cache-singleflight
s3-select
sub
tcp_read
test-reverting-lock-table
test_udp
testing
testing-sdx-generation
tikv
track-mount-e2e
upgrade-versions-to-4.00
volume_buffered_writes
worker-execute-ec-tasks
0.72
0.72.release
0.73
0.74
0.75
0.76
0.77
0.90
0.91
0.92
0.93
0.94
0.95
0.96
0.97
0.98
0.99
1.00
1.01
1.02
1.03
1.04
1.05
1.06
1.07
1.08
1.09
1.10
1.11
1.12
1.14
1.15
1.16
1.17
1.18
1.19
1.20
1.21
1.22
1.23
1.24
1.25
1.26
1.27
1.28
1.29
1.30
1.31
1.32
1.33
1.34
1.35
1.36
1.37
1.38
1.40
1.41
1.42
1.43
1.44
1.45
1.46
1.47
1.48
1.49
1.50
1.51
1.52
1.53
1.54
1.55
1.56
1.57
1.58
1.59
1.60
1.61
1.61RC
1.62
1.63
1.64
1.65
1.66
1.67
1.68
1.69
1.70
1.71
1.72
1.73
1.74
1.75
1.76
1.77
1.78
1.79
1.80
1.81
1.82
1.83
1.84
1.85
1.86
1.87
1.88
1.90
1.91
1.92
1.93
1.94
1.95
1.96
1.97
1.98
1.99
1;70
2.00
2.01
2.02
2.03
2.04
2.05
2.06
2.07
2.08
2.09
2.10
2.11
2.12
2.13
2.14
2.15
2.16
2.17
2.18
2.19
2.20
2.21
2.22
2.23
2.24
2.25
2.26
2.27
2.28
2.29
2.30
2.31
2.32
2.33
2.34
2.35
2.36
2.37
2.38
2.39
2.40
2.41
2.42
2.43
2.47
2.48
2.49
2.50
2.51
2.52
2.53
2.54
2.55
2.56
2.57
2.58
2.59
2.60
2.61
2.62
2.63
2.64
2.65
2.66
2.67
2.68
2.69
2.70
2.71
2.72
2.73
2.74
2.75
2.76
2.77
2.78
2.79
2.80
2.81
2.82
2.83
2.84
2.85
2.86
2.87
2.88
2.89
2.90
2.91
2.92
2.93
2.94
2.95
2.96
2.97
2.98
2.99
3.00
3.01
3.02
3.03
3.04
3.05
3.06
3.07
3.08
3.09
3.10
3.11
3.12
3.13
3.14
3.15
3.16
3.18
3.19
3.20
3.21
3.22
3.23
3.24
3.25
3.26
3.27
3.28
3.29
3.30
3.31
3.32
3.33
3.34
3.35
3.36
3.37
3.38
3.39
3.40
3.41
3.42
3.43
3.44
3.45
3.46
3.47
3.48
3.50
3.51
3.52
3.53
3.54
3.55
3.56
3.57
3.58
3.59
3.60
3.61
3.62
3.63
3.64
3.65
3.66
3.67
3.68
3.69
3.71
3.72
3.73
3.74
3.75
3.76
3.77
3.78
3.79
3.80
3.81
3.82
3.83
3.84
3.85
3.86
3.87
3.88
3.89
3.90
3.91
3.92
3.93
3.94
3.95
3.96
3.97
3.98
3.99
4.00
4.01
4.02
4.03
dev
helm-3.65.1
v0.69
v0.70beta
v3.33
${ noResults }
2 Commits (b68b9c6dd60c0daaea2c6f65b547dbec24ae0244)
| Author | SHA1 | Message | Date |
|---|---|---|---|
chrislu
|
a5db489f82 |
fix: Add topic cache invalidation and auto-creation on metadata requests
Add InvalidateTopicExistsCache method to SeaweedMQHandlerInterface and impl ement cache refresh logic in metadata response handler. When a consumer requests metadata for a topic that doesn't appear in the cache (but was just created by a producer), force a fresh broker check and auto-create the topic if needed with default partitions. This fix attempts to address the consumer stalling issue by: 1. Invalidating stale cache entries before checking broker 2. Automatically creating topics on metadata requests (like Kafka's auto.create.topics.enable=true) 3. Returning topics to consumers more reliably However, testing shows consumers still can't find topics even after creation, suggesting a deeper issue with topic persistence or broker client communication. Added InvalidateTopicExistsCache to mock handler as no-op for testing. Note: Integration testing reveals that consumers get 'topic does not exist' errors even when producers successfully create topics. This suggests the real issue is either: - Topics created by producers aren't visible to broker client queries - Broker client TopicExists() doesn't work correctly - There's a race condition in topic creation/registration Requires further investigation of broker client implementation and SMQ topic persistence logic. |
2 months ago |
|
chrislu
|
3c74ae2151 |
test: Add comprehensive unit tests for offset/fetch pattern
Add detailed unit tests to verify sequential consumption pattern:
1. TestOffsetCommitFetchPattern: Core test for:
- Consumer reads messages 0-N
- Consumer commits offset N
- Consumer fetches messages starting from N+1
- No message loss or duplication
2. TestOffsetFetchAfterCommit: Tests the critical case where:
- Consumer commits offset 163
- Consumer should fetch offset 164 and get data (not empty)
- This is where consumers currently get stuck
3. TestOffsetPersistencePattern: Verifies:
- Offsets persist correctly across restarts
- Offset recovery works after rebalancing
- Next offset calculation is correct
4. TestOffsetCommitConsistency: Ensures:
- Offset commits are atomic
- No partial updates
5. TestFetchEmptyPartitionHandling: Validates:
- Empty partition behavior
- Consumer doesn't give up on empty fetch
- Retry logic works correctly
6. TestLongPollWithOffsetCommit: Ensures:
- Long-poll duration is NOT reported as throttle
- Verifies fix from commit
|
2 months ago |