chrislu
|
f8a22038c5
|
passing along publisher name
|
7 months ago |
|
chrislu
|
3622196881
|
go fmt
|
7 months ago |
|
chrislu
|
d218fe54fa
|
go fmt
|
7 months ago |
|
chrislu
|
b1871427c3
|
persist consumer group offset
1. use one follower
2. read write consumer group offset
|
7 months ago |
|
chrislu
|
4e3701df77
|
reset last error
|
8 months ago |
|
chrislu
|
0f35b3a4ea
|
add example to publish a record
|
8 months ago |
|
chrislu
|
68ade92d4f
|
publisher send schema
|
8 months ago |
|
chrislu
|
5324e4288a
|
rename parameter
|
8 months ago |
|
chrislu
|
66a878af39
|
add control message to data message
|
9 months ago |
|
chrislu
|
546ae87c39
|
send is_close message to broker
|
9 months ago |
|
chrislu
|
c9df613b6b
|
add publisher name for debugging
|
9 months ago |
|
chrislu
|
da31e9b939
|
correctly wait for the publisher to finish
|
9 months ago |
|
chrislu
|
7003ce7425
|
publisher is able to fully send data and receive acks
still needs to close the pipes cleanly
|
9 months ago |
|
chrislu
|
33ab6cfcf7
|
adjust error
|
10 months ago |
|
chrislu
|
ead27a4f59
|
publish init add followers
|
10 months ago |
|
chrislu
|
1b4484bf0a
|
go fmt
|
10 months ago |
|
Chris Lu
|
90fcde0e26
|
report EOF error
|
11 months ago |
|
Chris Lu
|
07028a4864
|
minor
|
11 months ago |
|
Chris Lu
|
4dc6681833
|
proxy requests to lock owner
|
11 months ago |
|
chrislu
|
0b2e5ddc7c
|
wait 3 seconds before shutting down publish client, to wait for all messages to be received
|
11 months ago |
|
chrislu
|
545d5fbdf6
|
unused code
|
11 months ago |
|
chrislu
|
a297247137
|
wait for each publish clients
|
11 months ago |
|
chrislu
|
b6c5e57c30
|
read will block if no items
|
11 months ago |
|
chrislu
|
0bf5424a2e
|
adjust visibility
|
11 months ago |
|
chrislu
|
dedfd31dfb
|
refactor
|
11 months ago |
|
chrislu
|
cbf750a31f
|
refactor
|
11 months ago |
|
chrislu
|
b51dfe2bff
|
wait for publishing clients
|
11 months ago |
|
chrislu
|
f8c55f101e
|
clean up
|
11 months ago |
|
chrislu
|
841fafd0a8
|
publish to input buffer
currently, the input buffer may not exist when start to publish
|
11 months ago |
|
chrislu
|
91af1f3069
|
schedule jobs
|
11 months ago |
|
chrislu
|
34a78ffad0
|
remove isForPublish from LookupTopicBrokers
also adds a return parameter: whether the topic exists or not
|
11 months ago |
|
chrislu
|
026c54a9bb
|
fix publisher
|
11 months ago |
|
chrislu
|
6a7a679137
|
passing partition generation timestamp
|
12 months ago |
|
chrislu
|
d0d24f1e40
|
rename functions
|
12 months ago |
|
chrislu
|
ee41dbb7fc
|
rename functions
|
12 months ago |
Chris Lu
|
580940bf82
|
Merge accumulated changes related to message queue (#5098)
* balance partitions on brokers
* prepare topic partition first and then publish, move partition
* purge unused APIs
* clean up
* adjust logs
* add BalanceTopics() grpc API
* configure topic
* configure topic command
* refactor
* repair missing partitions
* sequence of operations to ensure ordering
* proto to close publishers and consumers
* rename file
* topic partition versioned by unixTimeNs
* create local topic partition
* close publishers
* randomize the client name
* wait until no publishers
* logs
* close stop publisher channel
* send last ack
* comments
* comment
* comments
* support list of brokers
* add cli options
* Update .gitignore
* logs
* return io.eof directly
* refactor
* optionally create topic
* refactoring
* detect consumer disconnection
* sub client wait for more messages
* subscribe by time stamp
* rename
* rename to sub_balancer
* rename
* adjust comments
* rename
* fix compilation
* rename
* rename
* SubscriberToSubCoordinator
* sticky rebalance
* go fmt
* add tests
* balance partitions on brokers
* prepare topic partition first and then publish, move partition
* purge unused APIs
* clean up
* adjust logs
* add BalanceTopics() grpc API
* configure topic
* configure topic command
* refactor
* repair missing partitions
* sequence of operations to ensure ordering
* proto to close publishers and consumers
* rename file
* topic partition versioned by unixTimeNs
* create local topic partition
* close publishers
* randomize the client name
* wait until no publishers
* logs
* close stop publisher channel
* send last ack
* comments
* comment
* comments
* support list of brokers
* add cli options
* Update .gitignore
* logs
* return io.eof directly
* refactor
* optionally create topic
* refactoring
* detect consumer disconnection
* sub client wait for more messages
* subscribe by time stamp
* rename
* rename to sub_balancer
* rename
* adjust comments
* rename
* fix compilation
* rename
* rename
* SubscriberToSubCoordinator
* sticky rebalance
* go fmt
* add tests
* tracking topic=>broker
* merge
* comment
|
1 year ago |
|
chrislu
|
411bdda08d
|
fix compilation
|
1 year ago |
|
chrislu
|
a65088d739
|
broker report stats to balancer
|
1 year ago |
|
chrislu
|
89a1fd1751
|
Squashed commit of the following:
commit 4827425146
Author: chrislu <chris.lu@gmail.com>
Date: Sat Sep 16 15:05:38 2023 -0700
balancer works
commit 3b50139f68
Author: chrislu <chris.lu@gmail.com>
Date: Fri Sep 15 22:22:32 2023 -0700
comments
commit 7f685ce7ba
Author: chrislu <chris.lu@gmail.com>
Date: Fri Sep 15 22:20:05 2023 -0700
adjust APIs
commit 436d99443b
Author: chrislu <chris.lu@gmail.com>
Date: Thu Sep 14 23:49:05 2023 -0700
receive broker stats
commit b771fefa37
Merge: 0a851ec00 890881037
Author: chrislu <chris.lu@gmail.com>
Date: Wed Sep 13 00:03:47 2023 -0700
Merge branch 'master' into sub
commit 0a851ec00b
Author: chrislu <chris.lu@gmail.com>
Date: Sun Sep 10 22:01:25 2023 -0700
Create balancer.go
commit 39941edc0b
Author: chrislu <chris.lu@gmail.com>
Date: Thu Sep 7 23:55:19 2023 -0700
add publisher shutdown
commit 875f562779
Author: chrislu <chris.lu@gmail.com>
Date: Wed Sep 6 23:16:41 2023 -0700
server side send response at least once per second
commit 984b6c54cf
Author: chrislu <chris.lu@gmail.com>
Date: Wed Sep 6 23:15:29 2023 -0700
ack interval 128
commit 2492a45499
Author: chrislu <chris.lu@gmail.com>
Date: Wed Sep 6 22:39:46 2023 -0700
ack interval
commit ba67e6ca29
Author: chrislu <chris.lu@gmail.com>
Date: Mon Sep 4 21:43:50 2023 -0700
api for sub
commit 9e4f985698
Author: chrislu <chris.lu@gmail.com>
Date: Mon Sep 4 21:43:30 2023 -0700
publish, benchmark
commit cb470d44df
Author: chrislu <chris.lu@gmail.com>
Date: Fri Sep 1 00:36:51 2023 -0700
can pub and sub
commit 1eb2da46d5
Author: chrislu <chris.lu@gmail.com>
Date: Mon Aug 28 09:02:12 2023 -0700
connect and publish
commit 504ae8383a
Author: chrislu <chris.lu@gmail.com>
Date: Mon Aug 28 09:01:25 2023 -0700
protoc version
commit dbcba75271
Author: chrislu <chris.lu@gmail.com>
Date: Sun Aug 27 18:59:04 2023 -0700
rename to lookup
commit c9caf33119
Author: chrislu <chris.lu@gmail.com>
Date: Sun Aug 27 18:33:46 2023 -0700
move functions
commit 4d6c18d86f
Author: chrislu <chris.lu@gmail.com>
Date: Sun Aug 27 17:50:59 2023 -0700
pub sub initial tests
commit 4eb8e8624d
Author: chrislu <chris.lu@gmail.com>
Date: Sun Aug 27 13:14:39 2023 -0700
rename
commit 1990456670
Author: chrislu <chris.lu@gmail.com>
Date: Sun Aug 27 13:13:14 2023 -0700
sub
commit 905911853d
Author: chrislu <chris.lu@gmail.com>
Date: Sat Aug 26 13:39:21 2023 -0700
adjust proto
|
1 year ago |
|
chrislu
|
39941edc0b
|
add publisher shutdown
|
1 year ago |
|
chrislu
|
984b6c54cf
|
ack interval 128
|
1 year ago |
|
chrislu
|
9e4f985698
|
publish, benchmark
|
1 year ago |
|
chrislu
|
cb470d44df
|
can pub and sub
|
1 year ago |
|
chrislu
|
1eb2da46d5
|
connect and publish
|
1 year ago |
|
chrislu
|
4d6c18d86f
|
pub sub initial tests
|
1 year ago |
|
chrislu
|
1990456670
|
sub
|
1 year ago |
