dc784bf217
merge current message queue code changes ( #6201 )
* listing files to convert to parquet
* write parquet files
* save logs into parquet files
* pass by value
* compact logs into parquet format
* can skip existing files
* refactor
* refactor
* fix compilation
* when no partition found
* refactor
* add untested parquet file read
* rename package
* refactor
* rename files
* remove unused
* add merged log read func
* parquet wants to know the file size
* rewind by time
* pass in stop ts
* add stop ts
* adjust log
* minor
* adjust log
* skip .parquet files when reading message logs
* skip non message files
* Update subscriber_record.go
* send messages
* skip message data with only ts
* skip non log files
* update parquet-go package
* ensure a valid record type
* add new field to a record type
* Update read_parquet_to_log.go
* fix parquet file name generation
* separating reading parquet and logs
* add key field
* add skipped logs
* use in memory cache
* refactor
* refactor
* refactor
* refactor, and change compact log
* refactor
* rename
* refactor
* fix format
* prefix v to version directory
2 months ago
ffe908371d
chore(deps): bump github.com/aws/aws-sdk-go-v2/service/s3 from 1.61.2 to 1.66.2 ( #6197 )
chore(deps): bump github.com/aws/aws-sdk-go-v2/service/s3
Bumps [github.com/aws/aws-sdk-go-v2/service/s3](https://github.com/aws/aws-sdk-go-v2 ) from 1.61.2 to 1.66.2.
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases )
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/service/s3/v1.61.2...service/s3/v1.66.2 )
---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go-v2/service/s3
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
2 months ago
92366134e3
chore(deps): bump github.com/gocql/gocql from 1.6.0 to 1.7.0 ( #6196 )
Bumps [github.com/gocql/gocql](https://github.com/gocql/gocql ) from 1.6.0 to 1.7.0.
- [Release notes](https://github.com/gocql/gocql/releases )
- [Changelog](https://github.com/apache/cassandra-gocql-driver/blob/trunk/CHANGELOG.md )
- [Commits](https://github.com/gocql/gocql/compare/v1.6.0...v1.7.0 )
---
updated-dependencies:
- dependency-name: github.com/gocql/gocql
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2 months ago
e2d30cc3bc
chore(deps): bump gocloud.dev/pubsub/natspubsub from 0.39.0 to 0.40.0 ( #6198 )
Bumps [gocloud.dev/pubsub/natspubsub](https://github.com/google/go-cloud ) from 0.39.0 to 0.40.0.
- [Release notes](https://github.com/google/go-cloud/releases )
- [Commits](https://github.com/google/go-cloud/compare/v0.39.0...v0.40.0 )
---
updated-dependencies:
- dependency-name: gocloud.dev/pubsub/natspubsub
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2 months ago
d185e6beea
chore(deps): bump golang.org/x/tools from 0.25.0 to 0.26.0 ( #6199 )
Bumps [golang.org/x/tools](https://github.com/golang/tools ) from 0.25.0 to 0.26.0.
- [Release notes](https://github.com/golang/tools/releases )
- [Commits](https://github.com/golang/tools/compare/v0.25.0...v0.26.0 )
---
updated-dependencies:
- dependency-name: golang.org/x/tools
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2 months ago
f67b5b20ca
chore(deps): bump cloud.google.com/go/pubsub from 1.43.0 to 1.45.1 ( #6200 )
Bumps [cloud.google.com/go/pubsub](https://github.com/googleapis/google-cloud-go ) from 1.43.0 to 1.45.1.
- [Release notes](https://github.com/googleapis/google-cloud-go/releases )
- [Changelog](https://github.com/googleapis/google-cloud-go/blob/main/CHANGES.md )
- [Commits](https://github.com/googleapis/google-cloud-go/compare/pubsub/v1.43.0...pubsub/v1.45.1 )
---
updated-dependencies:
- dependency-name: cloud.google.com/go/pubsub
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2 months ago
a7973ed7d1
fix deadlock hang when broadcast to clients ( #6184 )
fix deadlock when broadcast to clients
when master thransfer leader, the old master will disconnect with all
filers and volumeServers, if the cluster is a big , the broadcast
messages may be more big than the max of the channel len 100, then if the
KeepConnect was not listen on the channel in disconnect, it will
deadlock. and the whole cluster will not serve!
2 months ago
0f2c3648dc
fix mount: cannot locate OSXFUSE
addressing https://github.com/seaweedfs/seaweedfs/discussions/6193
2 months ago
5bddf0c085
[shell] volume.balance collect volume servers by dc rack node ( #6191 )
* chore: balance by rack
* fix: rm check lock
* fix: selected racks
* fix: selected nodes
* fix: containts
* fix: one collectVolumeServersByDcRackNode
* fix: revert lock and add lock
* fix: panic test
* revert noLock
2 months ago
65fb8fad99
Update filer_multipart.go ( #6188 )
fix the #6177 bug
2 months ago
0060a2cf9c
Fix 6181/6182 ( #6183 )
* set larger buf size for LogBuffer
* jump to next day when no more entry found
* Update weed/filer/filer_notify_read.go
---------
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
2 months ago
c29c912bdc
fix format ( #6185 )
unitest weed/shell fail
2 months ago
ad6240f066
fix parameter
2 months ago
8802843511
pass in bytes buffer to avoid possible race condition
related to https://github.com/seaweedfs/seaweedfs/issues/6180
2 months ago
3feb66d0a1
adding more debug message
related to https://github.com/seaweedfs/seaweedfs/issues/6180
2 months ago
228946369c
3.79
2 months ago
9873b033d1
backward compatible vif loading
2 months ago
f17048d32a
chore(deps): bump github.com/redis/go-redis/v9 from 9.6.2 to 9.7.0 ( #6174 )
Bumps [github.com/redis/go-redis/v9](https://github.com/redis/go-redis ) from 9.6.2 to 9.7.0.
- [Release notes](https://github.com/redis/go-redis/releases )
- [Changelog](https://github.com/redis/go-redis/blob/master/CHANGELOG.md )
- [Commits](https://github.com/redis/go-redis/compare/v9.6.2...v9.7.0 )
---
updated-dependencies:
- dependency-name: github.com/redis/go-redis/v9
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2 months ago
cd789d869d
chore(deps): bump modernc.org/sqlite from 1.33.0 to 1.33.1 ( #6173 )
Bumps [modernc.org/sqlite](https://gitlab.com/cznic/sqlite ) from 1.33.0 to 1.33.1.
- [Commits](https://gitlab.com/cznic/sqlite/compare/v1.33.0...v1.33.1 )
---
updated-dependencies:
- dependency-name: modernc.org/sqlite
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2 months ago
1f9f6fae6b
chore(deps): bump cloud.google.com/go/storage from 1.43.0 to 1.45.0 ( #6172 )
Bumps [cloud.google.com/go/storage](https://github.com/googleapis/google-cloud-go ) from 1.43.0 to 1.45.0.
- [Release notes](https://github.com/googleapis/google-cloud-go/releases )
- [Changelog](https://github.com/googleapis/google-cloud-go/blob/main/CHANGES.md )
- [Commits](https://github.com/googleapis/google-cloud-go/compare/pubsub/v1.43.0...pubsub/v1.45.0 )
---
updated-dependencies:
- dependency-name: cloud.google.com/go/storage
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2 months ago
c74dca10cb
chore(deps): bump github.com/ydb-platform/ydb-go-sdk/v3 from 3.77.1 to 3.89.2 ( #6171 )
chore(deps): bump github.com/ydb-platform/ydb-go-sdk/v3
Bumps [github.com/ydb-platform/ydb-go-sdk/v3](https://github.com/ydb-platform/ydb-go-sdk ) from 3.77.1 to 3.89.2.
- [Release notes](https://github.com/ydb-platform/ydb-go-sdk/releases )
- [Changelog](https://github.com/ydb-platform/ydb-go-sdk/blob/master/CHANGELOG.md )
- [Commits](https://github.com/ydb-platform/ydb-go-sdk/compare/v3.77.1...v3.89.2 )
---
updated-dependencies:
- dependency-name: github.com/ydb-platform/ydb-go-sdk/v3
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2 months ago
0f55dd452d
chore(deps): bump google.golang.org/grpc from 1.67.0 to 1.67.1 ( #6170 )
Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go ) from 1.67.0 to 1.67.1.
- [Release notes](https://github.com/grpc/grpc-go/releases )
- [Commits](https://github.com/grpc/grpc-go/compare/v1.67.0...v1.67.1 )
---
updated-dependencies:
- dependency-name: google.golang.org/grpc
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2 months ago
9105c6bdd1
fix format
2 months ago
ebbb35d533
3.78
2 months ago
089d4316ef
ensure 2 volume space since actual need 1.4x volume size empty space
2 months ago
6e388e29c9
correcting free volume count, factor it during ec encoding to ensure enough disk space available
fix https://github.com/seaweedfs/seaweedfs/issues/6163
2 months ago
8d6189bcc5
adjust output format
2 months ago
2f3d820f52
rename proto field
This should not have any impact.
2 months ago
ae5bd0667a
rename proto field from DestroyTime to expire_at_sec
For TTL volume converted into EC volume, this change may leave the volumes staying.
2 months ago
1c2b10303a
remove X-Forwarded-For
fix https://github.com/seaweedfs/seaweedfs/issues/6136
2 months ago
a3b97bfe03
[helm] Allow setting notification.toml config ( #6159 )
* [helm] Allow setting notification.toml config
* [helm] Set notification config conditionally
2 months ago
73921ce4f6
adjust help message
2 months ago
9369a88c5c
stop renaming worm files ( #6154 )
* stop renaming worm file
Signed-off-by: lou <alex1988@outlook.com>
* update after review
Signed-off-by: lou <alex1988@outlook.com>
* Update weed/server/filer_server_handlers_write.go
---------
Signed-off-by: lou <alex1988@outlook.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
2 months ago
970059d4ce
typo
2 months ago
d38456dc06
do not delete collection metrics when unmount a volume
fix https://github.com/seaweedfs/seaweedfs/issues/6139
2 months ago
6c7824c813
consistent sorted tags elastic,gocdk,rclone,sqlite,tikv,ydb
2 months ago
0df4a4298b
adjust comment
2 months ago
ad8e547690
chore(deps): bump github.com/prometheus/client_golang from 1.20.4 to 1.20.5 ( #6153 )
chore(deps): bump github.com/prometheus/client_golang
Bumps [github.com/prometheus/client_golang](https://github.com/prometheus/client_golang ) from 1.20.4 to 1.20.5.
- [Release notes](https://github.com/prometheus/client_golang/releases )
- [Changelog](https://github.com/prometheus/client_golang/blob/main/CHANGELOG.md )
- [Commits](https://github.com/prometheus/client_golang/compare/v1.20.4...v1.20.5 )
---
updated-dependencies:
- dependency-name: github.com/prometheus/client_golang
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2 months ago
eac37e41ef
chore(deps): bump google.golang.org/protobuf from 1.34.2 to 1.35.1 ( #6152 )
Bumps google.golang.org/protobuf from 1.34.2 to 1.35.1.
---
updated-dependencies:
- dependency-name: google.golang.org/protobuf
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2 months ago
f57b108475
chore(deps): bump github.com/hanwen/go-fuse/v2 from 2.6.1 to 2.6.2 ( #6151 )
Bumps [github.com/hanwen/go-fuse/v2](https://github.com/hanwen/go-fuse ) from 2.6.1 to 2.6.2.
- [Commits](https://github.com/hanwen/go-fuse/compare/v2.6.1...v2.6.2 )
---
updated-dependencies:
- dependency-name: github.com/hanwen/go-fuse/v2
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2 months ago
a8780b8831
chore(deps): bump github.com/getsentry/sentry-go from 0.29.0 to 0.29.1 ( #6150 )
Bumps [github.com/getsentry/sentry-go](https://github.com/getsentry/sentry-go ) from 0.29.0 to 0.29.1.
- [Release notes](https://github.com/getsentry/sentry-go/releases )
- [Changelog](https://github.com/getsentry/sentry-go/blob/master/CHANGELOG.md )
- [Commits](https://github.com/getsentry/sentry-go/compare/v0.29.0...v0.29.1 )
---
updated-dependencies:
- dependency-name: github.com/getsentry/sentry-go
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2 months ago
c54cbc3787
chore(deps): bump github.com/aws/aws-sdk-go-v2/config from 1.27.33 to 1.28.0 ( #6149 )
chore(deps): bump github.com/aws/aws-sdk-go-v2/config
Bumps [github.com/aws/aws-sdk-go-v2/config](https://github.com/aws/aws-sdk-go-v2 ) from 1.27.33 to 1.28.0.
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases )
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/config/v1.27.33...v1.28.0 )
---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go-v2/config
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2 months ago
d04859b01c
Support specifying rack and dataCenter for filers in helm chart ( #6144 )
2 months ago
54b91ba2b2
fix permission check in put request ( #6142 )
2 months ago
1e13b6879c
fix(volume): to avoid duplicate write a same needle ( #6138 )
fix WriteNeedleBlob to avoid duplicate write a same needle
Co-authored-by: 邓书东 <shudong_deng@hhnb2024010108.intsig.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
2 months ago
f11567816e
Revert "fix: volume data integrity check ( #6118 )"
This reverts commit e12b9ab746
2 months ago
7bd638de47
Fix invalid metric name ( #6141 )
Replaced `SeaweedFS_filer_` with `SeaweedFS_filerStore_` because the metric name was not found.
2 months ago
5bd5a76d25
feat: add resource annotations ( #6134 )
2 months ago
9dc47f6b97
Support topologySpreadConstraints in helm chart ( #6132 )
2 months ago
e12b9ab746
fix: volume data integrity check ( #6118 )
Co-authored-by: wang wusong <wangwusong@virtaitech.com>
2 months ago
Chris Lu
dependabot[bot]
wyang
chrislu
Konstantin Lebedev
han
Bruce
Davide Calabrese
Guang Jiong Lou
Dominik Grygiel
dsd
Brad Murray
oldgiova
wusong