yudrywet
8833745fbc
chore: fix some typos in comments ( #5497 )
8 months ago
chrislu
68ade92d4f
publisher send schema
8 months ago
chrislu
5324e4288a
rename parameter
8 months ago
chrislu
66a878af39
add control message to data message
9 months ago
chrislu
546ae87c39
send is_close message to broker
9 months ago
chrislu
c9df613b6b
add publisher name for debugging
9 months ago
chrislu
205d2285f3
setup follower by publisher
* the subscriber would getOrGen a local partition and wait
* the publisher would getOrGen a local partition. If localPartition follower is not setup, and init message has follower info, it would create followers based on init.Messages.
9 months ago
chrislu
da31e9b939
correctly wait for the publisher to finish
9 months ago
chrislu
7003ce7425
publisher is able to fully send data and receive acks
still needs to close the pipes cleanly
9 months ago
chrislu
33ab6cfcf7
adjust error
10 months ago
chrislu
ead27a4f59
publish init add followers
10 months ago
chrislu
1b4484bf0a
go fmt
10 months ago
Chris Lu
16b4f9d9ab
subscriber find broker leader first
11 months ago
Chris Lu
90fcde0e26
report EOF error
11 months ago
Chris Lu
07028a4864
minor
11 months ago
Chris Lu
4dc6681833
proxy requests to lock owner
11 months ago
chrislu
0b2e5ddc7c
wait 3 seconds before shutting down publish client, to wait for all messages to be received
11 months ago
chrislu
545d5fbdf6
unused code
11 months ago
chrislu
a297247137
wait for each publish clients
11 months ago
chrislu
b6c5e57c30
read will block if no items
11 months ago
chrislu
0bf5424a2e
adjust visibility
11 months ago
chrislu
dedfd31dfb
refactor
11 months ago
chrislu
cbf750a31f
refactor
11 months ago
chrislu
b51dfe2bff
wait for publishing clients
11 months ago
chrislu
f8c55f101e
clean up
11 months ago
chrislu
841fafd0a8
publish to input buffer
currently, the input buffer may not exist when start to publish
11 months ago
chrislu
91af1f3069
schedule jobs
11 months ago
chrislu
34a78ffad0
remove isForPublish from LookupTopicBrokers
also adds a return parameter: whether the topic exists or not
11 months ago
chrislu
7e6497cc1c
adjust publisher subscriber
11 months ago
chrislu
026c54a9bb
fix publisher
11 months ago
chrislu
3aa3991f0f
adjust client side logs
12 months ago
chrislu
6a7a679137
passing partition generation timestamp
12 months ago
chrislu
49428a303b
add batch index for each memory buffer
12 months ago
chrislu
d0d24f1e40
rename functions
12 months ago
chrislu
aa5b6a8187
adjust logs
12 months ago
chrislu
496fc8fbbf
refactor
12 months ago
chrislu
531f854af2
rename functions
12 months ago
chrislu
f8787a9761
rename functions
12 months ago
chrislu
ddd0fde094
rename functions
12 months ago
chrislu
ee41dbb7fc
rename functions
12 months ago
chrislu
47a4963d7a
subscription start from specified timestamp
12 months ago
chrislu
35869b5c80
subscriber can be notified of the assignment change when topic is just configured
Next: Subscriber needs to read by the timestamp offset.
12 months ago
chrislu
8b4cd50aac
log errors
12 months ago
chrislu
32bc8d6a38
adjust wait time
12 months ago
chrislu
458ddbf919
clean up dead code
12 months ago
chrislu
2845230329
passing broker into the assignments
1 year ago
chrislu
093fdc1621
subscriber can get assignments
1 year ago
chrislu
c950a40aad
subscriber keep connected to the balancer
1 year ago
Chris Lu
580940bf82
Merge accumulated changes related to message queue ( #5098 )
* balance partitions on brokers
* prepare topic partition first and then publish, move partition
* purge unused APIs
* clean up
* adjust logs
* add BalanceTopics() grpc API
* configure topic
* configure topic command
* refactor
* repair missing partitions
* sequence of operations to ensure ordering
* proto to close publishers and consumers
* rename file
* topic partition versioned by unixTimeNs
* create local topic partition
* close publishers
* randomize the client name
* wait until no publishers
* logs
* close stop publisher channel
* send last ack
* comments
* comment
* comments
* support list of brokers
* add cli options
* Update .gitignore
* logs
* return io.eof directly
* refactor
* optionally create topic
* refactoring
* detect consumer disconnection
* sub client wait for more messages
* subscribe by time stamp
* rename
* rename to sub_balancer
* rename
* adjust comments
* rename
* fix compilation
* rename
* rename
* SubscriberToSubCoordinator
* sticky rebalance
* go fmt
* add tests
* balance partitions on brokers
* prepare topic partition first and then publish, move partition
* purge unused APIs
* clean up
* adjust logs
* add BalanceTopics() grpc API
* configure topic
* configure topic command
* refactor
* repair missing partitions
* sequence of operations to ensure ordering
* proto to close publishers and consumers
* rename file
* topic partition versioned by unixTimeNs
* create local topic partition
* close publishers
* randomize the client name
* wait until no publishers
* logs
* close stop publisher channel
* send last ack
* comments
* comment
* comments
* support list of brokers
* add cli options
* Update .gitignore
* logs
* return io.eof directly
* refactor
* optionally create topic
* refactoring
* detect consumer disconnection
* sub client wait for more messages
* subscribe by time stamp
* rename
* rename to sub_balancer
* rename
* adjust comments
* rename
* fix compilation
* rename
* rename
* SubscriberToSubCoordinator
* sticky rebalance
* go fmt
* add tests
* tracking topic=>broker
* merge
* comment
1 year ago
chrislu
bf13f3ced7
add subscriber coordinator
1 year ago