chrislu
270e91b0be
adds locking
9 months ago
chrislu
b6fd1ff4ce
refactor
9 months ago
chrislu
d218fe54fa
go fmt
12 months ago
chrislu
b1871427c3
persist consumer group offset
1. use one follower
2. read write consumer group offset
1 year ago
chrislu
8d5bb7420d
rename
1 year ago
chrislu
1f20178ded
subscriber receives partitions and dispatch to processors
1 year ago
chrislu
ccdd9cd8de
refactor
1 year ago
chrislu
f37c0d0d7a
comment out println
1 year ago
chrislu
f07875e8e1
send flush message to follower before shutting down logBuffer
1 year ago
chrislu
e568e742c9
renaming
1 year ago
chrislu
47975e75bf
purge old code
1 year ago
chrislu
237e68059d
shutdown follower if no clients, not only publishers
to avoid overhead when a publisher repeatedly send one message and close
1 year ago
chrislu
c9df613b6b
add publisher name for debugging
1 year ago
chrislu
ca4f89a6f6
simplify
1 year ago
chrislu
0d55bddc75
logs
1 year ago
chrislu
e3c8be7f89
LocalPartition shutdown
1 year ago
chrislu
ab9fc0ad70
simplify
1 year ago
chrislu
7463058299
change visibility
1 year ago
chrislu
dab5454332
publish and send to follower
1 year ago
chrislu
f249f831b1
MaybeShutdownLocalPartition refactor
1 year ago
chrislu
2a0ab7e90d
fix
1 year ago
chrislu
059a120708
refactor
1 year ago
chrislu
7d87c1d2bb
refactor
1 year ago
chrislu
5cc94a05b9
separate goroutine to send ack to publisher
1 year ago
chrislu
f67c8309d8
rename
1 year ago
chrislu
205d2285f3
setup follower by publisher
* the subscriber would getOrGen a local partition and wait
* the publisher would getOrGen a local partition. If localPartition follower is not setup, and init message has follower info, it would create followers based on init.Messages.
1 year ago
chrislu
6ef3e6bcad
logging
1 year ago
chrislu
8e5068fd2f
notify
1 year ago
chrislu
529fa8c4c8
go fmt
1 year ago
chrislu
64a32730a2
add size()
1 year ago
chrislu
6f75a0af55
ask follower to follow
1 year ago
chrislu
3b28433cb1
change LogBuffer visibility in LocalPartition
1 year ago
chrislu
1b4484bf0a
go fmt
1 year ago
chrislu
fdf0ea8e11
passing partition info
1 year ago
chrislu
3795d8dca8
release local topic partition if no publisher and subscribers
1 year ago
chrislu
ba73199174
reuse local partition
1 year ago
chrislu
fa59a5d67e
read from disk if not in memory
1 year ago
chrislu
45994641e9
lint
1 year ago
chrislu
6a7a679137
passing partition generation timestamp
1 year ago
chrislu
d51efddf5c
flush to disk
Need to do: read from disk
1 year ago
chrislu
49428a303b
add batch index for each memory buffer
1 year ago
chrislu
093fdc1621
subscriber can get assignments
1 year ago
Chris Lu
580940bf82
Merge accumulated changes related to message queue ( #5098 )
* balance partitions on brokers
* prepare topic partition first and then publish, move partition
* purge unused APIs
* clean up
* adjust logs
* add BalanceTopics() grpc API
* configure topic
* configure topic command
* refactor
* repair missing partitions
* sequence of operations to ensure ordering
* proto to close publishers and consumers
* rename file
* topic partition versioned by unixTimeNs
* create local topic partition
* close publishers
* randomize the client name
* wait until no publishers
* logs
* close stop publisher channel
* send last ack
* comments
* comment
* comments
* support list of brokers
* add cli options
* Update .gitignore
* logs
* return io.eof directly
* refactor
* optionally create topic
* refactoring
* detect consumer disconnection
* sub client wait for more messages
* subscribe by time stamp
* rename
* rename to sub_balancer
* rename
* adjust comments
* rename
* fix compilation
* rename
* rename
* SubscriberToSubCoordinator
* sticky rebalance
* go fmt
* add tests
* balance partitions on brokers
* prepare topic partition first and then publish, move partition
* purge unused APIs
* clean up
* adjust logs
* add BalanceTopics() grpc API
* configure topic
* configure topic command
* refactor
* repair missing partitions
* sequence of operations to ensure ordering
* proto to close publishers and consumers
* rename file
* topic partition versioned by unixTimeNs
* create local topic partition
* close publishers
* randomize the client name
* wait until no publishers
* logs
* close stop publisher channel
* send last ack
* comments
* comment
* comments
* support list of brokers
* add cli options
* Update .gitignore
* logs
* return io.eof directly
* refactor
* optionally create topic
* refactoring
* detect consumer disconnection
* sub client wait for more messages
* subscribe by time stamp
* rename
* rename to sub_balancer
* rename
* adjust comments
* rename
* fix compilation
* rename
* rename
* SubscriberToSubCoordinator
* sticky rebalance
* go fmt
* add tests
* tracking topic=>broker
* merge
* comment
1 year ago
chrislu
bf13f3ced7
add subscriber coordinator
2 years ago
chrislu
734178093e
refactor TopicPartition struct
2 years ago
chrislu
23f334dd13
comment
2 years ago
chrislu
c7e05e4e71
ensure latest stats are reported
2 years ago
chrislu
1492bf7552
fix listing topics
2 years ago
chrislu
6c6655de9d
wip
2 years ago
chrislu
a65088d739
broker report stats to balancer
2 years ago