Chris Lu
1cd2e64aac
merge chunks during upload ( #4130 )
* merge chunks during upload
* fix test
2 years ago
chrislu
2452f3b2f7
sort chunks by offset
2 years ago
chrislu
361d97941b
mysql: Missed escaping column name
fix https://github.com/seaweedfs/seaweedfs/issues/4129
2 years ago
chrislu
efcffe4f77
no need to set chunks during flush
2 years ago
chrislu
85f1b70207
reduce group size from 256MB to 64MB
2 years ago
chrislu
86657ed3a8
avoid repeatedly adding manifest chunks
2 years ago
chrislu
28fe578944
avoid possible nil entry
2 years ago
chrislu
47fa5ef979
Merge branch 'master' of https://github.com/seaweedfs/seaweedfs
2 years ago
chrislu
9e0d526163
fix compilation
2 years ago
chrislu
7f49c59c14
cluster.ps add filer meta sync progress
2 years ago
chrislu
340e7c3a2e
chunk group remove manifestChunks and reset sections in SetChunks()
2 years ago
chrislu
e650c8397d
ensure section.chunks is always garbage free
2 years ago
chrislu
75bdd4a0d1
refactor
2 years ago
dependabot[bot]
44b0176e9e
build(deps): bump github.com/aws/aws-sdk-go from 1.44.171 to 1.44.175 ( #4126 )
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go ) from 1.44.171 to 1.44.175.
- [Release notes](https://github.com/aws/aws-sdk-go/releases )
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.44.171...v1.44.175 )
---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2 years ago
dependabot[bot]
f030186e3a
build(deps): bump github.com/google/flatbuffers from 22.11.23+incompatible to 23.1.4+incompatible ( #4125 )
build(deps): bump github.com/google/flatbuffers
Bumps [github.com/google/flatbuffers](https://github.com/google/flatbuffers ) from 22.11.23+incompatible to 23.1.4+incompatible.
- [Release notes](https://github.com/google/flatbuffers/releases )
- [Changelog](https://github.com/google/flatbuffers/blob/master/CHANGELOG.md )
- [Commits](https://github.com/google/flatbuffers/compare/v22.11.23...v23.1.4 )
---
updated-dependencies:
- dependency-name: github.com/google/flatbuffers
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2 years ago
dependabot[bot]
36f81bd462
build(deps): bump golang.org/x/sys from 0.3.0 to 0.4.0 ( #4123 )
Bumps [golang.org/x/sys](https://github.com/golang/sys ) from 0.3.0 to 0.4.0.
- [Release notes](https://github.com/golang/sys/releases )
- [Commits](https://github.com/golang/sys/compare/v0.3.0...v0.4.0 )
---
updated-dependencies:
- dependency-name: golang.org/x/sys
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2 years ago
dependabot[bot]
d0de2e2dfc
build(deps): bump github.com/tikv/client-go/v2 from 2.0.3 to 2.0.4 ( #4124 )
Bumps [github.com/tikv/client-go/v2](https://github.com/tikv/client-go ) from 2.0.3 to 2.0.4.
- [Release notes](https://github.com/tikv/client-go/releases )
- [Commits](https://github.com/tikv/client-go/compare/v2.0.3...v2.0.4 )
---
updated-dependencies:
- dependency-name: github.com/tikv/client-go/v2
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2 years ago
dependabot[bot]
cc0abaf0f9
build(deps): bump actions/dependency-review-action from 3.0.2 to 3.0.3 ( #4121 )
Bumps [actions/dependency-review-action](https://github.com/actions/dependency-review-action ) from 3.0.2 to 3.0.3.
- [Release notes](https://github.com/actions/dependency-review-action/releases )
- [Commits](0ff3da6f81
...c090f4e553673e6e505ea70d6a95362ee12adb94)
---
updated-dependencies:
- dependency-name: actions/dependency-review-action
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2 years ago
dependabot[bot]
8abace77d9
build(deps): bump actions/checkout from 3.2.0 to 3.3.0 ( #4120 )
Bumps [actions/checkout](https://github.com/actions/checkout ) from 3.2.0 to 3.3.0.
- [Release notes](https://github.com/actions/checkout/releases )
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md )
- [Commits](755da8c3cf
...ac593985615ec2ede58e132d2e21d2b1cbd6127c)
---
updated-dependencies:
- dependency-name: actions/checkout
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2 years ago
dependabot[bot]
9b42942cca
build(deps): bump google.golang.org/api from 0.105.0 to 0.106.0 ( #4122 )
Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client ) from 0.105.0 to 0.106.0.
- [Release notes](https://github.com/googleapis/google-api-go-client/releases )
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md )
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.105.0...v0.106.0 )
---
updated-dependencies:
- dependency-name: google.golang.org/api
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2 years ago
Guo Lei
83e29faa55
use time.NewTicker instead of time.Tick. ( #4119 )
2 years ago
chrislu
51d4a4b28d
3.39
2 years ago
chrislu
0b52f08226
Merge branch 'master' of https://github.com/seaweedfs/seaweedfs
2 years ago
chrislu
469c959e6a
add missing options for webdav
2 years ago
monchickey
3e2c9ea73d
Add image cropping. ( #4117 )
2 years ago
chrislu
296fdc296c
mount: faster add chunks
2 years ago
chrislu
49b84d5866
Merge branch 'master' of https://github.com/seaweedfs/seaweedfs
2 years ago
chrislu
2abf817580
fix for stream reader
fix https://github.com/seaweedfs/seaweedfs/issues/4112
2 years ago
Tobias Gurtzick
9ffe1d6aec
add code of conduct ( #4109 )
Signed-off-by: Tobias Gurtzick <magic@wizardtales.com>
Signed-off-by: Tobias Gurtzick <magic@wizardtales.com>
2 years ago
chrislu
e367444586
add notes for full version
2 years ago
chrislu
5423790b2c
add back previous chunk upload selection algo
2 years ago
chrislu
7bdae5172e
batch delete EC needles
fix https://github.com/seaweedfs/seaweedfs/issues/4107
2 years ago
chrislu
8aec430df7
Merge branch 'master' of https://github.com/seaweedfs/seaweedfs
2 years ago
chrislu
c87a3ffce3
avoid hard coded versions
2 years ago
zemul
6b4c033431
add mount log ( #4101 )
* filer.backup use replication.source.filer
* add mount log
* Revert "filer.backup use replication.source.filer"
This reverts commit 07bf6f956c
.
* fix
Co-authored-by: zemul <zhouzemiao@ihuman.com>
2 years ago
chrislu
48f2edc065
3.38
2 years ago
Chris Lu
d4566d4aaa
more solid weed mount ( #4089 )
* compare chunks by timestamp
* fix slab clearing error
* fix test compilation
* move oldest chunk to sealed, instead of by fullness
* lock on fh.entryViewCache
* remove verbose logs
* revert slat clearing
* less logs
* less logs
* track write and read by timestamp
* remove useless logic
* add entry lock on file handle release
* use mem chunk only, swap file chunk has problems
* comment out code that maybe used later
* add debug mode to compare data read and write
* more efficient readResolvedChunks with linked list
* small optimization
* fix test compilation
* minor fix on writer
* add SeparateGarbageChunks
* group chunks into sections
* turn off debug mode
* fix tests
* fix tests
* tmp enable swap file chunk
* Revert "tmp enable swap file chunk"
This reverts commit 985137ec47
.
* simple refactoring
* simple refactoring
* do not re-use swap file chunk. Sealed chunks should not be re-used.
* comment out debugging facilities
* either mem chunk or swap file chunk is fine now
* remove orderedMutex as *semaphore.Weighted
not found impactful
* optimize size calculation for changing large files
* optimize performance to avoid going through the long list of chunks
* still problems with swap file chunk
* rename
* tiny optimization
* swap file chunk save only successfully read data
* fix
* enable both mem and swap file chunk
* resolve chunks with range
* rename
* fix chunk interval list
* also change file handle chunk group when adding chunks
* pick in-active chunk with time-decayed counter
* fix compilation
* avoid nil with empty fh.entry
* refactoring
* rename
* rename
* refactor visible intervals to *list.List
* refactor chunkViews to *list.List
* add IntervalList for generic interval list
* change visible interval to use IntervalList in generics
* cahnge chunkViews to *IntervalList[*ChunkView]
* use NewFileChunkSection to create
* rename variables
* refactor
* fix renaming leftover
* renaming
* renaming
* add insert interval
* interval list adds lock
* incrementally add chunks to readers
Fixes:
1. set start and stop offset for the value object
2. clone the value object
3. use pointer instead of copy-by-value when passing to interval.Value
4. use insert interval since adding chunk could be out of order
* fix tests compilation
* fix tests compilation
2 years ago
dependabot[bot]
367353b936
build(deps): bump github.com/aws/aws-sdk-go from 1.44.167 to 1.44.171 ( #4100 )
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go ) from 1.44.167 to 1.44.171.
- [Release notes](https://github.com/aws/aws-sdk-go/releases )
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.44.167...v1.44.171 )
---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2 years ago
dependabot[bot]
de787ce4ab
build(deps): bump github.com/ydb-platform/ydb-go-sdk/v3 from 3.41.0 to 3.42.1 ( #4099 )
build(deps): bump github.com/ydb-platform/ydb-go-sdk/v3
Bumps [github.com/ydb-platform/ydb-go-sdk/v3](https://github.com/ydb-platform/ydb-go-sdk ) from 3.41.0 to 3.42.1.
- [Release notes](https://github.com/ydb-platform/ydb-go-sdk/releases )
- [Changelog](https://github.com/ydb-platform/ydb-go-sdk/blob/master/CHANGELOG.md )
- [Commits](https://github.com/ydb-platform/ydb-go-sdk/compare/v3.41.0...v3.42.1 )
---
updated-dependencies:
- dependency-name: github.com/ydb-platform/ydb-go-sdk/v3
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2 years ago
dependabot[bot]
04bba1a6b2
build(deps): bump modernc.org/sqlite from 1.20.0 to 1.20.1 ( #4098 )
---
updated-dependencies:
- dependency-name: modernc.org/sqlite
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2 years ago
CommanderRoot
c7c9d22f37
filer.store.mysql: Use utf8mb4 instead of 3 byte UTF8 ( #4094 )
2 years ago
CommanderRoot
c2280e94cf
filer.store.mysql: Replace deprecated upsert syntax ( #4096 )
2 years ago
CommanderRoot
265a56630b
filer.store.mysql: Escape table columns in SQL query ( #4095 )
2 years ago
Guo Lei
ed5f3f073b
add more help message, in case of misunderstanding ( #4092 )
2 years ago
Guo Lei
d8cfa1552b
support enable/disable vacuum ( #4087 )
* stop vacuum
* suspend/resume vacuum
* remove unused code
* rename
* rename param
2 years ago
dependabot[bot]
84492976c9
build(deps): bump github.com/aws/aws-sdk-go from 1.44.162 to 1.44.167 ( #4085 )
2 years ago
dependabot[bot]
bd176359cb
build(deps): bump github.com/prometheus/procfs from 0.8.0 to 0.9.0 ( #4084 )
2 years ago
dependabot[bot]
b2aa1a23ff
build(deps): bump github.com/arangodb/go-driver from 1.4.0 to 1.4.1 ( #4083 )
2 years ago
Guo Lei
5f993a5f20
Fix ide warning ( #4082 )
fix warning message from IDE
"redundant type from array, slice, or map composite literal"
2 years ago
fibbanachi
3901f2da4d
Add logtostderr option in helm chart ( #4080 )
2 years ago