Tree: 21aaddf665

add-admin-and-worker-to-helm-charts

add-ec-vacuum

add-foundation-db

add_fasthttp_client

add_remote_storage

adding-message-queue-integration-tests

avoid_releasing_temp_file_on_write

changing-to-zap

collect-public-metrics

create-table-snapshot-api-design

data_query_pushdown

dependabot/maven/other/java/client/com.google.protobuf-protobuf-java-3.25.5

dependabot/maven/other/java/examples/org.apache.hadoop-hadoop-common-3.4.0

detect-and-plan-ec-tasks

do-not-retry-if-error-is-NotFound

enhance-erasure-coding

fallback-on-volume-id-lookup

fasthttp

filer1_maintenance_branch

fix-GetObjectLockConfigurationHandler

fix-versioning-listing-only

ftp

gh-pages

improve-fuse-mount

improve-fuse-mount2

logrus

master

message_send

mount2

mq-subscribe

mq2

original_weed_mount

random_access_file

refactor-needle-read-operations

refactor-volume-write

remote_overlay

revert-5134-patch-1

revert-5819-patch-1

revert-6434-bugfix-missing-s3-audit

s3-select

sub

tcp_read

test-reverting-lock-table

test_udp

testing

testing-sdx-generation

tikv

track-mount-e2e

volume_buffered_writes

worker-execute-ec-tasks

0.72

0.72.release

0.73

0.74

0.75

0.76

0.77

0.90

0.91

0.92

0.93

0.94

0.95

0.96

0.97

0.98

0.99

1.00

1.01

1.02

1.03

1.04

1.05

1.06

1.07

1.08

1.09

1.10

1.11

1.12

1.14

1.15

1.16

1.17

1.18

1.19

1.20

1.21

1.22

1.23

1.24

1.25

1.26

1.27

1.28

1.29

1.30

1.31

1.32

1.33

1.34

1.35

1.36

1.37

1.38

1.40

1.41

1.42

1.43

1.44

1.45

1.46

1.47

1.48

1.49

1.50

1.51

1.52

1.53

1.54

1.55

1.56

1.57

1.58

1.59

1.60

1.61

1.61RC

1.62

1.63

1.64

1.65

1.66

1.67

1.68

1.69

1.70

1.71

1.72

1.73

1.74

1.75

1.76

1.77

1.78

1.79

1.80

1.81

1.82

1.83

1.84

1.85

1.86

1.87

1.88

1.90

1.91

1.92

1.93

1.94

1.95

1.96

1.97

1.98

1.99

1;70

2.00

2.01

2.02

2.03

2.04

2.05

2.06

2.07

2.08

2.09

2.10

2.11

2.12

2.13

2.14

2.15

2.16

2.17

2.18

2.19

2.20

2.21

2.22

2.23

2.24

2.25

2.26

2.27

2.28

2.29

2.30

2.31

2.32

2.33

2.34

2.35

2.36

2.37

2.38

2.39

2.40

2.41

2.42

2.43

2.47

2.48

2.49

2.50

2.51

2.52

2.53

2.54

2.55

2.56

2.57

2.58

2.59

2.60

2.61

2.62

2.63

2.64

2.65

2.66

2.67

2.68

2.69

2.70

2.71

2.72

2.73

2.74

2.75

2.76

2.77

2.78

2.79

2.80

2.81

2.82

2.83

2.84

2.85

2.86

2.87

2.88

2.89

2.90

2.91

2.92

2.93

2.94

2.95

2.96

2.97

2.98

2.99

3.00

3.01

3.02

3.03

3.04

3.05

3.06

3.07

3.08

3.09

3.10

3.11

3.12

3.13

3.14

3.15

3.16

3.18

3.19

3.20

3.21

3.22

3.23

3.24

3.25

3.26

3.27

3.28

3.29

3.30

3.31

3.32

3.33

3.34

3.35

3.36

3.37

3.38

3.39

3.40

3.41

3.42

3.43

3.44

3.45

3.46

3.47

3.48

3.50

3.51

3.52

3.53

3.54

3.55

3.56

3.57

3.58

3.59

3.60

3.61

3.62

3.63

3.64

3.65

3.66

3.67

3.68

3.69

3.71

3.72

3.73

3.74

3.75

3.76

3.77

3.78

3.79

3.80

3.81

3.82

3.83

3.84

3.85

3.86

3.87

3.88

3.89

3.90

3.91

3.92

3.93

3.94

3.95

3.96

3.97

3.98

3.99

dev

helm-3.65.1

v0.69

v0.70beta

v3.33

Branches Tags

${ item.name }

Create tag ${ searchTerm }

Create branch ${ searchTerm }

from '21aaddf665'

${ noResults }

Commit Graph

73 Commits (21aaddf665985bc89b8908ea018bcd0071f8fa4c)

Author	SHA1	Message	Date
Chris Lu	51543bbb87	Admin UI: Add message queue to admin UI (#6958) * add a menu item "Message Queue" * add a menu item "Message Queue" * move the "brokers" link under it. * add "topics", "subscribers". Add pages for them. * refactor * show topic details * admin display publisher and subscriber info * remove publisher and subscribers from the topic row pull down * collecting more stats from publishers and subscribers * fix layout * fix publisher name * add local listeners for mq broker and agent * render consumer group offsets * remove subscribers from left menu * topic with retention * support editing topic retention * show retention when listing topics * create bucket * Update s3_buckets_templ.go * embed the static assets into the binary fix https://github.com/seaweedfs/seaweedfs/issues/6964	4 months ago
chrislu	96632a34b1	add version to volume proto	5 months ago
Chris Lu	02773a6107	Accumulated changes for message queue (#6600) * rename * set agent address * refactor * add agent sub * pub messages * grpc new client * can publish records via agent * send init message with session id * fmt * check cancelled request while waiting * use sessionId * handle possible nil stream * subscriber process messages * separate debug port * use atomic int64 * less logs * minor * skip io.EOF * rename * remove unused * use saved offsets * do not reuse session, since always session id is new after restart remove last active ts from SessionEntry * simplify printing * purge unused * just proxy the subscription, skipping the session step * adjust offset types * subscribe offset type and possible value * start after the known tsns * avoid wrongly set startPosition * move * remove * refactor * typo * fix * fix changed path	8 months ago
Chris Lu	cc05874d06	Add message queue agent (#6463) * scaffold message queue agent * adjust proto, add mq_agent * add agent client implementation * remove unused function * agent publish server implementation * adding agent	9 months ago
chrislu	ae5bd0667a	rename proto field from DestroyTime to expire_at_sec For TTL volume converted into EC volume, this change may leave the volumes staying.	1 year ago
chrislu	f8a22038c5	passing along publisher name	1 year ago
chrislu	edc87c16f9	balance subscribers need to ensure the offsets are persisted	1 year ago
chrislu	d40b350f54	coordinator receives unassignment ack	1 year ago
chrislu	2190d59bc3	refactor ToDo 1. handle unassignment on client side 2. get notified of completed unassignment 3. in consumer_group.go, balance and send unassignment	1 year ago
chrislu	3b7f24b1fc	unused	1 year ago
chrislu	d8ab22012a	track offset	1 year ago
chrislu	c6db3f31a1	start consuming ASAP	1 year ago
chrislu	b1871427c3	persist consumer group offset 1. use one follower 2. read write consumer group offset	1 year ago
chrislu	94742b8ace	add subscribe follower	1 year ago
chrislu	372bd8d71d	consumer instance passing MaxPartitionCount to coordinator	1 year ago
chrislu	04fb4c34e3	add proto for value	2 years ago
chrislu	66a878af39	add control message to data message	2 years ago
chrislu	546ae87c39	send is_close message to broker	2 years ago
chrislu	c9df613b6b	add publisher name for debugging	2 years ago
chrislu	ab9fc0ad70	simplify	2 years ago
chrislu	205d2285f3	setup follower by publisher * the subscriber would getOrGen a local partition and wait * the publisher would getOrGen a local partition. If localPartition follower is not setup, and init message has follower info, it would create followers based on init.Messages.	2 years ago
chrislu	7003ce7425	publisher is able to fully send data and receive acks still needs to close the pipes cleanly	2 years ago
chrislu	516cba7053	remove FollowInMemoryMessages	2 years ago
chrislu	6f75a0af55	ask follower to follow	2 years ago
chrislu	7e443ef0a1	latest protoc-gen-go	2 years ago
chrislu	ead27a4f59	publish init add followers	2 years ago
chrislu	1b4484bf0a	go fmt	2 years ago
Chris Lu	e27deed4bc	upgrade protoc	2 years ago
chrislu	34a78ffad0	remove isForPublish from LookupTopicBrokers also adds a return parameter: whether the topic exists or not	2 years ago
chrislu	49428a303b	add batch index for each memory buffer	2 years ago
chrislu	e8611ed85d	subscribe with partition offset	2 years ago
chrislu	ee41dbb7fc	rename functions	2 years ago
chrislu	1fed37bed0	add offset	2 years ago
chrislu	093fdc1621	subscriber can get assignments	2 years ago
chrislu	c950a40aad	subscriber keep connected to the balancer	2 years ago
Chris Lu	580940bf82	Merge accumulated changes related to message queue (#5098) * balance partitions on brokers * prepare topic partition first and then publish, move partition * purge unused APIs * clean up * adjust logs * add BalanceTopics() grpc API * configure topic * configure topic command * refactor * repair missing partitions * sequence of operations to ensure ordering * proto to close publishers and consumers * rename file * topic partition versioned by unixTimeNs * create local topic partition * close publishers * randomize the client name * wait until no publishers * logs * close stop publisher channel * send last ack * comments * comment * comments * support list of brokers * add cli options * Update .gitignore * logs * return io.eof directly * refactor * optionally create topic * refactoring * detect consumer disconnection * sub client wait for more messages * subscribe by time stamp * rename * rename to sub_balancer * rename * adjust comments * rename * fix compilation * rename * rename * SubscriberToSubCoordinator * sticky rebalance * go fmt * add tests * balance partitions on brokers * prepare topic partition first and then publish, move partition * purge unused APIs * clean up * adjust logs * add BalanceTopics() grpc API * configure topic * configure topic command * refactor * repair missing partitions * sequence of operations to ensure ordering * proto to close publishers and consumers * rename file * topic partition versioned by unixTimeNs * create local topic partition * close publishers * randomize the client name * wait until no publishers * logs * close stop publisher channel * send last ack * comments * comment * comments * support list of brokers * add cli options * Update .gitignore * logs * return io.eof directly * refactor * optionally create topic * refactoring * detect consumer disconnection * sub client wait for more messages * subscribe by time stamp * rename * rename to sub_balancer * rename * adjust comments * rename * fix compilation * rename * rename * SubscriberToSubCoordinator * sticky rebalance * go fmt * add tests * tracking topic=>broker * merge * comment	2 years ago
chrislu	deb86ff4a6	upgrading grpc	2 years ago
chrislu	de0b969b36	Revert "rename" This reverts commit 35b5264ab7.	2 years ago
chrislu	35b5264ab7	rename	2 years ago
chrislu	ebd6f96d35	adjust mq.proto	2 years ago
chrislu	d7c211fe8c	adjust mq proto	2 years ago
chrislu	9d589b48e6	rename function	2 years ago
chrislu	3cf9b8d621	create topic and report topic	2 years ago
chrislu	b3f94feede	do create topic	2 years ago
chrislu	0361c321b4	add CreateTopic API	2 years ago
chrislu	6c6655de9d	wip	2 years ago
chrislu	340aea9b34	add broker leader	2 years ago
chrislu	a65088d739	broker report stats to balancer	2 years ago
chrislu	89a1fd1751	Squashed commit of the following: commit 4827425146 Author: chrislu <chris.lu@gmail.com> Date: Sat Sep 16 15:05:38 2023 -0700 balancer works commit 3b50139f68 Author: chrislu <chris.lu@gmail.com> Date: Fri Sep 15 22:22:32 2023 -0700 comments commit 7f685ce7ba Author: chrislu <chris.lu@gmail.com> Date: Fri Sep 15 22:20:05 2023 -0700 adjust APIs commit 436d99443b Author: chrislu <chris.lu@gmail.com> Date: Thu Sep 14 23:49:05 2023 -0700 receive broker stats commit b771fefa37 Merge: 0a851ec00 890881037 Author: chrislu <chris.lu@gmail.com> Date: Wed Sep 13 00:03:47 2023 -0700 Merge branch 'master' into sub commit 0a851ec00b Author: chrislu <chris.lu@gmail.com> Date: Sun Sep 10 22:01:25 2023 -0700 Create balancer.go commit 39941edc0b Author: chrislu <chris.lu@gmail.com> Date: Thu Sep 7 23:55:19 2023 -0700 add publisher shutdown commit 875f562779 Author: chrislu <chris.lu@gmail.com> Date: Wed Sep 6 23:16:41 2023 -0700 server side send response at least once per second commit 984b6c54cf Author: chrislu <chris.lu@gmail.com> Date: Wed Sep 6 23:15:29 2023 -0700 ack interval 128 commit 2492a45499 Author: chrislu <chris.lu@gmail.com> Date: Wed Sep 6 22:39:46 2023 -0700 ack interval commit ba67e6ca29 Author: chrislu <chris.lu@gmail.com> Date: Mon Sep 4 21:43:50 2023 -0700 api for sub commit 9e4f985698 Author: chrislu <chris.lu@gmail.com> Date: Mon Sep 4 21:43:30 2023 -0700 publish, benchmark commit cb470d44df Author: chrislu <chris.lu@gmail.com> Date: Fri Sep 1 00:36:51 2023 -0700 can pub and sub commit 1eb2da46d5 Author: chrislu <chris.lu@gmail.com> Date: Mon Aug 28 09:02:12 2023 -0700 connect and publish commit 504ae8383a Author: chrislu <chris.lu@gmail.com> Date: Mon Aug 28 09:01:25 2023 -0700 protoc version commit dbcba75271 Author: chrislu <chris.lu@gmail.com> Date: Sun Aug 27 18:59:04 2023 -0700 rename to lookup commit c9caf33119 Author: chrislu <chris.lu@gmail.com> Date: Sun Aug 27 18:33:46 2023 -0700 move functions commit 4d6c18d86f Author: chrislu <chris.lu@gmail.com> Date: Sun Aug 27 17:50:59 2023 -0700 pub sub initial tests commit 4eb8e8624d Author: chrislu <chris.lu@gmail.com> Date: Sun Aug 27 13:14:39 2023 -0700 rename commit 1990456670 Author: chrislu <chris.lu@gmail.com> Date: Sun Aug 27 13:13:14 2023 -0700 sub commit 905911853d Author: chrislu <chris.lu@gmail.com> Date: Sat Aug 26 13:39:21 2023 -0700 adjust proto	2 years ago
chrislu	4827425146	balancer works	2 years ago