chrislu
270e91b0be
adds locking
4 months ago
chrislu
b6fd1ff4ce
refactor
4 months ago
chrislu
d218fe54fa
go fmt
7 months ago
chrislu
b1871427c3
persist consumer group offset
1. use one follower
2. read write consumer group offset
7 months ago
chrislu
8d5bb7420d
rename
7 months ago
chrislu
1f20178ded
subscriber receives partitions and dispatch to processors
7 months ago
chrislu
ccdd9cd8de
refactor
9 months ago
chrislu
f37c0d0d7a
comment out println
9 months ago
chrislu
f07875e8e1
send flush message to follower before shutting down logBuffer
9 months ago
chrislu
e568e742c9
renaming
9 months ago
chrislu
47975e75bf
purge old code
9 months ago
chrislu
237e68059d
shutdown follower if no clients, not only publishers
to avoid overhead when a publisher repeatedly send one message and close
9 months ago
chrislu
c9df613b6b
add publisher name for debugging
9 months ago
chrislu
ca4f89a6f6
simplify
9 months ago
chrislu
0d55bddc75
logs
9 months ago
chrislu
e3c8be7f89
LocalPartition shutdown
9 months ago
chrislu
ab9fc0ad70
simplify
9 months ago
chrislu
7463058299
change visibility
9 months ago
chrislu
dab5454332
publish and send to follower
9 months ago
chrislu
f249f831b1
MaybeShutdownLocalPartition refactor
9 months ago
chrislu
2a0ab7e90d
fix
9 months ago
chrislu
059a120708
refactor
9 months ago
chrislu
7d87c1d2bb
refactor
9 months ago
chrislu
5cc94a05b9
separate goroutine to send ack to publisher
9 months ago
chrislu
f67c8309d8
rename
9 months ago
chrislu
205d2285f3
setup follower by publisher
* the subscriber would getOrGen a local partition and wait
* the publisher would getOrGen a local partition. If localPartition follower is not setup, and init message has follower info, it would create followers based on init.Messages.
9 months ago
chrislu
6ef3e6bcad
logging
9 months ago
chrislu
8e5068fd2f
notify
9 months ago
chrislu
529fa8c4c8
go fmt
9 months ago
chrislu
64a32730a2
add size()
9 months ago
chrislu
6f75a0af55
ask follower to follow
10 months ago
chrislu
3b28433cb1
change LogBuffer visibility in LocalPartition
10 months ago
chrislu
1b4484bf0a
go fmt
10 months ago
chrislu
fdf0ea8e11
passing partition info
11 months ago
chrislu
3795d8dca8
release local topic partition if no publisher and subscribers
11 months ago
chrislu
ba73199174
reuse local partition
11 months ago
chrislu
fa59a5d67e
read from disk if not in memory
11 months ago
chrislu
45994641e9
lint
12 months ago
chrislu
6a7a679137
passing partition generation timestamp
12 months ago
chrislu
d51efddf5c
flush to disk
Need to do: read from disk
12 months ago
chrislu
49428a303b
add batch index for each memory buffer
12 months ago
chrislu
093fdc1621
subscriber can get assignments
1 year ago
Chris Lu
580940bf82
Merge accumulated changes related to message queue ( #5098 )
* balance partitions on brokers
* prepare topic partition first and then publish, move partition
* purge unused APIs
* clean up
* adjust logs
* add BalanceTopics() grpc API
* configure topic
* configure topic command
* refactor
* repair missing partitions
* sequence of operations to ensure ordering
* proto to close publishers and consumers
* rename file
* topic partition versioned by unixTimeNs
* create local topic partition
* close publishers
* randomize the client name
* wait until no publishers
* logs
* close stop publisher channel
* send last ack
* comments
* comment
* comments
* support list of brokers
* add cli options
* Update .gitignore
* logs
* return io.eof directly
* refactor
* optionally create topic
* refactoring
* detect consumer disconnection
* sub client wait for more messages
* subscribe by time stamp
* rename
* rename to sub_balancer
* rename
* adjust comments
* rename
* fix compilation
* rename
* rename
* SubscriberToSubCoordinator
* sticky rebalance
* go fmt
* add tests
* balance partitions on brokers
* prepare topic partition first and then publish, move partition
* purge unused APIs
* clean up
* adjust logs
* add BalanceTopics() grpc API
* configure topic
* configure topic command
* refactor
* repair missing partitions
* sequence of operations to ensure ordering
* proto to close publishers and consumers
* rename file
* topic partition versioned by unixTimeNs
* create local topic partition
* close publishers
* randomize the client name
* wait until no publishers
* logs
* close stop publisher channel
* send last ack
* comments
* comment
* comments
* support list of brokers
* add cli options
* Update .gitignore
* logs
* return io.eof directly
* refactor
* optionally create topic
* refactoring
* detect consumer disconnection
* sub client wait for more messages
* subscribe by time stamp
* rename
* rename to sub_balancer
* rename
* adjust comments
* rename
* fix compilation
* rename
* rename
* SubscriberToSubCoordinator
* sticky rebalance
* go fmt
* add tests
* tracking topic=>broker
* merge
* comment
1 year ago
chrislu
bf13f3ced7
add subscriber coordinator
1 year ago
chrislu
734178093e
refactor TopicPartition struct
1 year ago
chrislu
23f334dd13
comment
1 year ago
chrislu
c7e05e4e71
ensure latest stats are reported
1 year ago
chrislu
1492bf7552
fix listing topics
1 year ago
chrislu
6c6655de9d
wip
1 year ago
chrislu
a65088d739
broker report stats to balancer
1 year ago