Tree: 662a6ac8ee

add-ec-vacuum

add_fasthttp_client

add_remote_storage

adding-message-queue-integration-tests

adjust-fsck-cutoff-default

also-delete-parent-directory-if-empty

avoid_releasing_temp_file_on_write

changing-to-zap

collect-public-metrics

copilot/fix-helm-chart-installation

copilot/fix-s3-object-tagging-issue

copilot/sub-pr-7677

create-table-snapshot-api-design

data_query_pushdown

dependabot/maven/other/java/client/com.google.protobuf-protobuf-java-3.25.5

dependabot/maven/other/java/examples/org.apache.hadoop-hadoop-common-3.4.0

detect-and-plan-ec-tasks

do-not-retry-if-error-is-NotFound

ec-disk-type-support

enhance-erasure-coding

fasthttp

filer1_maintenance_branch

fix-GetObjectLockConfigurationHandler

fix-mount-http-parallelism

fix-mount-read-throughput-7504

fix-s3-object-tagging-issue-7589

fix-versioning-listing-only

ftp

gh-pages

improve-fuse-mount

improve-fuse-mount2

logrus

master

message_send

mount2

mq-subscribe

mq2

nfs-cookie-prefix-list-fixes

optimize-delete-lookups

original_weed_mount

pr-7412

random_access_file

refactor-needle-read-operations

refactor-volume-write

remote_overlay

revert-5134-patch-1

revert-5819-patch-1

revert-6434-bugfix-missing-s3-audit

s3-remote-cache-singleflight

s3-select

sub

tcp_read

test-reverting-lock-table

test_udp

testing

testing-sdx-generation

tikv

track-mount-e2e

upgrade-versions-to-4.00

volume_buffered_writes

worker-execute-ec-tasks

0.72

0.72.release

0.73

0.74

0.75

0.76

0.77

0.90

0.91

0.92

0.93

0.94

0.95

0.96

0.97

0.98

0.99

1.00

1.01

1.02

1.03

1.04

1.05

1.06

1.07

1.08

1.09

1.10

1.11

1.12

1.14

1.15

1.16

1.17

1.18

1.19

1.20

1.21

1.22

1.23

1.24

1.25

1.26

1.27

1.28

1.29

1.30

1.31

1.32

1.33

1.34

1.35

1.36

1.37

1.38

1.40

1.41

1.42

1.43

1.44

1.45

1.46

1.47

1.48

1.49

1.50

1.51

1.52

1.53

1.54

1.55

1.56

1.57

1.58

1.59

1.60

1.61

1.61RC

1.62

1.63

1.64

1.65

1.66

1.67

1.68

1.69

1.70

1.71

1.72

1.73

1.74

1.75

1.76

1.77

1.78

1.79

1.80

1.81

1.82

1.83

1.84

1.85

1.86

1.87

1.88

1.90

1.91

1.92

1.93

1.94

1.95

1.96

1.97

1.98

1.99

1;70

2.00

2.01

2.02

2.03

2.04

2.05

2.06

2.07

2.08

2.09

2.10

2.11

2.12

2.13

2.14

2.15

2.16

2.17

2.18

2.19

2.20

2.21

2.22

2.23

2.24

2.25

2.26

2.27

2.28

2.29

2.30

2.31

2.32

2.33

2.34

2.35

2.36

2.37

2.38

2.39

2.40

2.41

2.42

2.43

2.47

2.48

2.49

2.50

2.51

2.52

2.53

2.54

2.55

2.56

2.57

2.58

2.59

2.60

2.61

2.62

2.63

2.64

2.65

2.66

2.67

2.68

2.69

2.70

2.71

2.72

2.73

2.74

2.75

2.76

2.77

2.78

2.79

2.80

2.81

2.82

2.83

2.84

2.85

2.86

2.87

2.88

2.89

2.90

2.91

2.92

2.93

2.94

2.95

2.96

2.97

2.98

2.99

3.00

3.01

3.02

3.03

3.04

3.05

3.06

3.07

3.08

3.09

3.10

3.11

3.12

3.13

3.14

3.15

3.16

3.18

3.19

3.20

3.21

3.22

3.23

3.24

3.25

3.26

3.27

3.28

3.29

3.30

3.31

3.32

3.33

3.34

3.35

3.36

3.37

3.38

3.39

3.40

3.41

3.42

3.43

3.44

3.45

3.46

3.47

3.48

3.50

3.51

3.52

3.53

3.54

3.55

3.56

3.57

3.58

3.59

3.60

3.61

3.62

3.63

3.64

3.65

3.66

3.67

3.68

3.69

3.71

3.72

3.73

3.74

3.75

3.76

3.77

3.78

3.79

3.80

3.81

3.82

3.83

3.84

3.85

3.86

3.87

3.88

3.89

3.90

3.91

3.92

3.93

3.94

3.95

3.96

3.97

3.98

3.99

4.00

4.01

4.02

4.03

dev

helm-3.65.1

v0.69

v0.70beta

v3.33

Branches Tags

${ item.name }

Create tag ${ searchTerm }

Create branch ${ searchTerm }

from '662a6ac8ee'

${ noResults }

Commit Graph

67 Commits (662a6ac8ee45caede35b82f4c5527cc30c0de0c2)

Author	SHA1	Message	Date
Chris Lu	84b8a8e010	filer.sync: fix checkpoint not being saved properly (#7719) * filer.sync: fix race condition on first checkpoint save Initialize lastWriteTime to time.Now() instead of zero time to prevent the first checkpoint save from being triggered immediately when the first event arrives. This gives async jobs time to complete and update the watermark before the checkpoint is saved. Previously, the zero time caused lastWriteTime.Add(3s).Before(now) to be true on the first event, triggering an immediate checkpoint save attempt. But since jobs are processed asynchronously, the watermark was still 0 (initial value), causing the save to be skipped due to the 'if offsetTsNs == 0 { return nil }' check. Fixes #7717 * filer.sync: save checkpoint on graceful shutdown Add graceful shutdown handling to save the final checkpoint when filer.sync is terminated. Previously, any sync progress within the last 3-second checkpoint interval would be lost on shutdown. Changes: - Add syncState struct to track current processor and offset save info - Add atomic pointers syncStateA2B and syncStateB2A for both directions - Register grace.OnInterrupt hook to save checkpoints on shutdown - Modify doSubscribeFilerMetaChanges to update sync state atomically This ensures that when filer.sync is restarted, it resumes from the correct position instead of potentially replaying old events. Fixes #7717	6 days ago
Numblgw	aebf3952b7	filer sync: source path and exclude path support dir suffix (#6268) filer sync: source path and exclude paht support dir suffix Co-authored-by: liguowei <liguowei@xinye.com>	1 year ago
Max Denushev	a5fe6e21bc	feat(filer.backup): add ignore errors option (#6235) * feat(filer.backup): add ignore errors option * feat(filer.backup): fix 404 error wrap * feat(filer.backup): fix wrapping function * feat(filer.backup): fix wrapping errors in genProcessFunction * Update weed/command/filer_backup.go * Update weed/command/filer_backup.go * Update weed/command/filer_backup.go --------- Co-authored-by: Max Denushev <denushev@tochka.com> Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>	1 year ago
chrislu	19d14b4c71	minor	1 year ago
vadimartynov	b796c21fa9	Added loadSecurityConfigOnce (#5792)	1 year ago
chrislu	3a82f5ffad	ensure metadata follow a specific folder fix https://github.com/seaweedfs/seaweedfs/issues/5774	1 year ago
Konstantin Lebedev	d821cb3b18	fix: sync without dir /buckets/some/.uploads/hash_hash (#5402)	2 years ago
chrislu	83e4b02517	fix	2 years ago
chrislu	15b66a6633	refactor	2 years ago
XIAOYQ	be166b434f	fix: skip s3 .uploads	2 years ago
Konstantin Lebedev	8d23e36c45	fix: doDeleteFiles deletes files (#5198)	2 years ago
Konstantin Lebedev	1169f94310	Fix filer sync set offset (#5197) * fix: compose 2mount with sync * fix: DATA RACE https://github.com/seaweedfs/seaweedfs/issues/5194 https://github.com/seaweedfs/seaweedfs/issues/5195	2 years ago
Konstantin Lebedev	b9d32d32e1	chore: filer sync add doDeleteFiles option for create only mode (#5166)	2 years ago
Konstantin Lebedev	2b229e98ce	fix: add doDeleteFile option for filer backup	2 years ago
Konstantin Lebedev	3c5295a1a6	filer backup add option for exclude file names that match the regexp to sync on filer	2 years ago
chrislu	5db9fcccd4	refactoring	3 years ago
chrislu	81fdf3651b	grpc connection to filer add sw-client-id header	3 years ago
chrislu	8a40fa8993	more detailed logs	3 years ago
Jiffs Maverick	4b0430e71d	[metrics] Add the ability to control bind ip (#4012)	3 years ago
chrislu	21c0587900	go fmt	3 years ago
Ryan Russell	2c92a9ff74	refactor: `DefaultConcurrencyLimit` var rename (#3658)	3 years ago
bernardx	228b133afa	new 'concurrency' parameter for filer.sync (#3579) Co-authored-by: XIAOYQ <xiaoyq@eudic.net>	3 years ago
qzh	7fcfaf7bc9	fix(filer.sync): offset may be set to 0 (#3451) * fix(filer.sync): initializing the offset is related to the path * fix(filer.sync): the offset maybe to be set to 0. Co-authored-by: zhihao.qu <zhihao.qu@ly.com>	3 years ago
qzh	400f0c3e5d	fix(filer.sync): initializing the offset is related to the path (#3450) Co-authored-by: zhihao.qu <zhihao.qu@ly.com>	3 years ago
chrislu	67814a5c79	refactor and fix strings.Split	3 years ago
chrislu	1a4bf0dcb5	filer.sync: parallelize the filer.sync	3 years ago
chrislu	0e9478488d	filer.sync: fix when excluded paths is empty	3 years ago
chrislu	26dbc6c905	move to https://github.com/seaweedfs/seaweedfs	3 years ago
Konstantin Lebedev	7e09a548a6	exclude directories to sync on filer	3 years ago
chrislu	64f3d6fb6e	metadata subscription uses client epoch	3 years ago
chrislu	4a65159250	fix reading time	4 years ago
zhihao.qu	4d0d1848c6	fix(filer.sync): modify clientName format : from -> to	4 years ago
zhihao.qu	42d04c581b	feat(filer.sync): add metricsServer in filer.sync. Metrics include: (1) the offset of the filer.sync (2) the last send timestamp of the filer subscription	4 years ago
zhihao.qu	14d82c3dea	feat(filer.sync): add offset to path.	4 years ago
zhihao.qu	cd5cca36a4	feat(filer.sync): add fromTsMs. Extract signature from doSubscribeFilerMetaChanges	4 years ago
creeew	02ae102731	fix filer.sync missing source srv uploaded files to target when target down	4 years ago
chrislu	a2b101a737	subscribe metadata between a range	4 years ago
chrislu	1384529eb7	Fix filer.backup deletes files in backup folder in incremental mode fix https://github.com/chrislusf/seaweedfs/issues/2919	4 years ago
chrislu	202a29d014	refactoring	4 years ago
chrislu	6a12520a96	fix logging	4 years ago
chrislu	5c87fcc6d2	add client id for all metadata listening clients	4 years ago
chrislu	9f9ef1340c	use streaming mode for long poll grpc calls streaming mode would create separate grpc connections for each call. this is to ensure the long poll connections are properly closed.	4 years ago
Chris Lu	e5fc35ed0c	change server address from string to a type	4 years ago
Chris Lu	8e125339d5	line wrap	4 years ago
Chris Lu	68ea99190f	add additionalPathPrefixes	4 years ago
Chris Lu	c5f38c365d	go fmt	4 years ago
Chris Lu	df85f7a1eb	adjust help message	4 years ago
Chris Lu	6b743dbbf9	refactor client subscribe metadata	4 years ago
Chris Lu	3afbf04007	add TLS grpc support for filer meta clients	4 years ago
Chris Lu	c2e0a75c1f	adjust logs	5 years ago