You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

Tree: d26c260041

add-ec-vacuum

add_fasthttp_client

add_remote_storage

adding-message-queue-integration-tests

adjust-fsck-cutoff-default

also-delete-parent-directory-if-empty

avoid_releasing_temp_file_on_write

changing-to-zap

collect-public-metrics

copilot/fix-helm-chart-installation

copilot/fix-s3-object-tagging-issue

copilot/sub-pr-7677

create-table-snapshot-api-design

data_query_pushdown

dependabot/maven/other/java/client/com.google.protobuf-protobuf-java-3.25.5

dependabot/maven/other/java/examples/org.apache.hadoop-hadoop-common-3.4.0

detect-and-plan-ec-tasks

do-not-retry-if-error-is-NotFound

ec-disk-type-support

enhance-erasure-coding

fasthttp

filer1_maintenance_branch

fix-GetObjectLockConfigurationHandler

fix-mount-http-parallelism

fix-mount-read-throughput-7504

fix-s3-object-tagging-issue-7589

fix-versioning-listing-only

ftp

gh-pages

improve-fuse-mount

improve-fuse-mount2

logrus

master

message_send

mount2

mq-subscribe

mq2

nfs-cookie-prefix-list-fixes

optimize-delete-lookups

original_weed_mount

pr-7412

raft-dual-write

random_access_file

refactor-needle-read-operations

refactor-volume-write

remote_overlay

remove-implicit-directory-handling

revert-5134-patch-1

revert-5819-patch-1

revert-6434-bugfix-missing-s3-audit

s3-remote-cache-singleflight

s3-select

sub

tcp_read

test-reverting-lock-table

test_udp

testing

testing-sdx-generation

tikv

track-mount-e2e

upgrade-versions-to-4.00

volume_buffered_writes

worker-execute-ec-tasks

0.72

0.72.release

0.73

0.74

0.75

0.76

0.77

0.90

0.91

0.92

0.93

0.94

0.95

0.96

0.97

0.98

0.99

1.00

1.01

1.02

1.03

1.04

1.05

1.06

1.07

1.08

1.09

1.10

1.11

1.12

1.14

1.15

1.16

1.17

1.18

1.19

1.20

1.21

1.22

1.23

1.24

1.25

1.26

1.27

1.28

1.29

1.30

1.31

1.32

1.33

1.34

1.35

1.36

1.37

1.38

1.40

1.41

1.42

1.43

1.44

1.45

1.46

1.47

1.48

1.49

1.50

1.51

1.52

1.53

1.54

1.55

1.56

1.57

1.58

1.59

1.60

1.61

1.61RC

1.62

1.63

1.64

1.65

1.66

1.67

1.68

1.69

1.70

1.71

1.72

1.73

1.74

1.75

1.76

1.77

1.78

1.79

1.80

1.81

1.82

1.83

1.84

1.85

1.86

1.87

1.88

1.90

1.91

1.92

1.93

1.94

1.95

1.96

1.97

1.98

1.99

1;70

2.00

2.01

2.02

2.03

2.04

2.05

2.06

2.07

2.08

2.09

2.10

2.11

2.12

2.13

2.14

2.15

2.16

2.17

2.18

2.19

2.20

2.21

2.22

2.23

2.24

2.25

2.26

2.27

2.28

2.29

2.30

2.31

2.32

2.33

2.34

2.35

2.36

2.37

2.38

2.39

2.40

2.41

2.42

2.43

2.47

2.48

2.49

2.50

2.51

2.52

2.53

2.54

2.55

2.56

2.57

2.58

2.59

2.60

2.61

2.62

2.63

2.64

2.65

2.66

2.67

2.68

2.69

2.70

2.71

2.72

2.73

2.74

2.75

2.76

2.77

2.78

2.79

2.80

2.81

2.82

2.83

2.84

2.85

2.86

2.87

2.88

2.89

2.90

2.91

2.92

2.93

2.94

2.95

2.96

2.97

2.98

2.99

3.00

3.01

3.02

3.03

3.04

3.05

3.06

3.07

3.08

3.09

3.10

3.11

3.12

3.13

3.14

3.15

3.16

3.18

3.19

3.20

3.21

3.22

3.23

3.24

3.25

3.26

3.27

3.28

3.29

3.30

3.31

3.32

3.33

3.34

3.35

3.36

3.37

3.38

3.39

3.40

3.41

3.42

3.43

3.44

3.45

3.46

3.47

3.48

3.50

3.51

3.52

3.53

3.54

3.55

3.56

3.57

3.58

3.59

3.60

3.61

3.62

3.63

3.64

3.65

3.66

3.67

3.68

3.69

3.71

3.72

3.73

3.74

3.75

3.76

3.77

3.78

3.79

3.80

3.81

3.82

3.83

3.84

3.85

3.86

3.87

3.88

3.89

3.90

3.91

3.92

3.93

3.94

3.95

3.96

3.97

3.98

3.99

4.00

4.01

4.02

4.03

dev

helm-3.65.1

v0.69

v0.70beta

v3.33

Branches Tags

${ item.name }

Create tag ${ searchTerm }

Create branch ${ searchTerm }

from 'd26c260041'

${ noResults }

seaweedfs/weed

History

Chris Lu d26c260041 s3: fix memory leak in ListObjectVersions with early termination (#7785) * s3: fix memory leak in ListObjectVersions with early termination This fixes a critical memory leak in S3 versioned bucket listing operations: 1. Add maxCollect parameter to findVersionsRecursively() to stop collecting versions once we have enough for the response + truncation detection 2. Add early termination checks throughout the recursive traversal to prevent scanning entire buckets when only a small number of results are requested 3. Use clear() on tracking maps after collection to help GC reclaim memory 4. Create new slice with exact capacity when truncating results instead of re-slicing, which allows GC to reclaim the excess backing array memory 5. Pre-allocate result slice with reasonable initial capacity to reduce reallocations during collection Before this fix, listing versions on a bucket with many objects and versions would load ALL versions into memory before pagination, causing OOM crashes. Fixes memory exhaustion when calling ListObjectVersions on large versioned buckets. * s3: fix pre-allocation capacity to be consistent with maxCollect Address review feedback: the previous capping logic caused an inconsistency where initialCap was capped to 1000 but maxCollect was maxKeys+1, leading to unnecessary reallocations when maxKeys was 1000. Fix by: 1. Cap maxKeys to 1000 (S3 API limit) at the start of the function 2. Use maxKeys+1 directly for slice capacity, ensuring consistency with the maxCollect parameter passed to findVersionsRecursively		4 days ago
..
admin	Fix admin GUI list ordering on refresh (#7782)	4 days ago
cluster	feat: add S3 bucket size and object count metrics (#7776)	5 days ago
command	filer: improve FoundationDB performance by disabling batch by default (#7770)	5 days ago
credential	fix: Admin UI user creation fails before filer discovery (#7624) (#7625)	2 weeks ago
filer	filer: improve FoundationDB performance by disabling batch by default (#7770)	5 days ago
filer_client	Clean up logs and deprecated functions (#7339)	2 months ago
glog	Add Kafka Gateway (#7231)	2 months ago
iam	feat(iam): add SetUserStatus and UpdateAccessKey actions (#7750)	6 days ago
iamapi	Refactor: Extract common IAM logic into shared weed/iam package (#7747)	6 days ago
images	Migrates from disintegration/imaging c2019 to cognusion/imaging c2024. (#5533)	2 years ago
kms	S3 API: Add integration with KMS providers (#7152)	4 months ago
mount	mount: add periodic metadata sync to protect chunks from orphan cleanup (#7700)	1 week ago
mq	mount: improve read throughput with parallel chunk fetching (#7569)	3 weeks ago
notification	Fix webhook duplicate deliveries and POST to GET conversion (#7668)	2 weeks ago
operation	fix: cache successful volume lookups instead of failed ones (#7698)	1 week ago
pb	feat(iam): add SetUserStatus and UpdateAccessKey actions (#7750)	6 days ago
query	mount: improve read throughput with parallel chunk fetching (#7569)	3 weeks ago
remote_storage	Filer: Fixed critical bugs in the Azure SDK migration (PR #7310) (#7401)	2 months ago
replication	Filer: Fixed critical bugs in the Azure SDK migration (PR #7310) (#7401)	2 months ago
s3api	s3: fix memory leak in ListObjectVersions with early termination (#7785)	4 days ago
security	remove spoof-able request header (#7103)	5 months ago
sequence	remove unused function	2 years ago
server	fix: correctly detect missing source file during volume copy (#7784)	4 days ago
sftpd	SFTP: support reloading user store on HUP signal (#7651)	2 weeks ago
shell	Humanize output for `weed.server` by default (#7758)	5 days ago
static	Fix Broken Links (#5287)	2 years ago
stats	feat: add S3 bucket size and object count metrics (#7776)	5 days ago
storage	Fix uncleanable size=0 orphans with volume.fsck -forcePurging (#7783)	4 days ago
telemetry	convert error fromating to %w everywhere (#6995)	5 months ago
topology	fix random volume ids in master.html (#7655)	2 weeks ago
util	4.03	5 days ago
wdclient	fix: WaitUntilConnected now respects context cancellation during sleep (#7737)	7 days ago
worker	Add disk-aware EC rebalancing (#7597)	3 weeks ago
Makefile	Update README and weed/Makefile (#7571)	3 weeks ago
weed.go	set exit status	9 months ago