41cbbefb0b
move ec task logic
4 months ago
ffd91383c6
worker can run ec work and report status
but:
1. one volume should not be repeatedly worked on.
2. ec shards needs to be distributed and source data should be deleted.
4 months ago
19df3fcc5d
worker register itself first
4 months ago
a3d727034a
clean up logs
4 months ago
c557ec0062
worker reconnect to admin
4 months ago
63817fe7ee
test canScheduleTaskNow
4 months ago
afdca513ac
sticky worker id
4 months ago
14e95186d6
skip task ID column
4 months ago
960acc031e
rendering pending tasks
4 months ago
6cf78754b2
build and restart
4 months ago
01e8cc4600
worker retries connecting to admin
4 months ago
e7084c5b7e
compiles
4 months ago
16a29b57f9
compiles, detection uses volumeSizeLimitMB from master
4 months ago
a1966e9692
working worker and admin. Task detection is not working yet.
4 months ago
f10c6305dc
Update .gitignore
4 months ago
cd28ce7786
admin start wtih grpc port. worker has its own working directory
4 months ago
2941cdf95e
tasks are assigned
4 months ago
688a91039d
generate write load
4 months ago
167ab29eb6
start master, volume servers, filer
Current Status
✅ Master: Healthy and running (port 9333)
✅ Filer: Healthy and running (port 8888)
✅ Volume Servers: All 6 servers running (ports 8080-8085)
🔄 Admin/Workers: Will start when dependencies are ready
4 months ago
a4cf55a741
remove "enhanced" reference
4 months ago
30ad65bb5f
integration test. ec worker task
4 months ago
0d8e1f8a17
reorganized the codebase to move the simulation framework and tests into their own dedicated package
4 months ago
a0874d201c
added simulation as tests
4 months ago
add122484c
initial design
4 months ago
7ab85c3748
return proper default value for locking and versioning
fix https://github.com/seaweedfs/seaweedfs/issues/6971
fix https://github.com/seaweedfs/seaweedfs/issues/7028
4 months ago
4f72a1778f
minor
4 months ago
2c5ffe16cf
Fix all in one deployment ( #7031 )
* make maxVolumes configurable for allInOne deployment
Signed-off-by: Mohamed Sekour <mohamed.sekour@exfo.com>
* Update all-in-one-deployment.yaml
fix typo
* add robustness
---------
Signed-off-by: Mohamed Sekour <mohamed.sekour@exfo.com>
4 months ago
5ac037f763
change priority of admin credentials from env varaibles ( #7032 )
* change priority of admin credentials from env varaibles
* address comment
4 months ago
dd464cd339
use latest v3.18.4
4 months ago
8531326b55
adding admin credential
4 months ago
e3d3c495ab
S3 API: simpler way to start s3 with credentials ( #7030 )
* simpler way to start s3 with credentials
* AWS_ACCESS_KEY_ID=access_key AWS_SECRET_ACCESS_KEY=secret_key weed s3
* last adding credentials from env variables
* Update weed/s3api/auth_credentials.go
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
* simplify
* adjust doc
---------
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
4 months ago
d5085cd1f7
newer helm version
fix https://github.com/seaweedfs/seaweedfs/issues/7029
4 months ago
a81421f393
chore(deps): bump gocloud.dev from 0.42.0 to 0.43.0 ( #7023 )
---
updated-dependencies:
- dependency-name: gocloud.dev
dependency-version: 0.43.0
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
5 months ago
33b9017b48
fix listing objects ( #7008 )
* fix listing objects
* add more list testing
* address comments
* fix next marker
* fix isTruncated in listing
* fix tests
* address tests
* Update s3api_object_handlers_multipart.go
* fixes
* store json into bucket content, for tagging and cors
* switch bucket metadata from json to proto
* fix
* Update s3api_bucket_config.go
* fix test issue
* fix test_bucket_listv2_delimiter_prefix
* Update cors.go
* skip special characters
* passing listing
* fix test_bucket_list_delimiter_prefix
* ok. fix the xsd generated go code now
* fix cors tests
* fix test
* fix test_bucket_list_unordered and test_bucket_listv2_unordered
do not accept the allow-unordered and delimiter parameter combination
* fix test_bucket_list_objects_anonymous and test_bucket_listv2_objects_anonymous
The tests test_bucket_list_objects_anonymous and test_bucket_listv2_objects_anonymous were failing because they try to set bucket ACL to public-read, but SeaweedFS only supported private ACL.
Updated PutBucketAclHandler to use the existing ExtractAcl function which already supports all standard S3 canned ACLs
Replaced the hardcoded check for only private ACL with proper ACL parsing that handles public-read, public-read-write, authenticated-read, bucket-owner-read, bucket-owner-full-control, etc.
Added unit tests to verify all standard canned ACLs are accepted
* fix list unordered
The test is expecting the error code to be InvalidArgument instead of InvalidRequest
* allow anonymous listing( and head, get)
* fix test_bucket_list_maxkeys_invalid
Invalid values: max-keys=blah → Returns ErrInvalidMaxKeys (HTTP 400)
* updating IsPublicRead when parsing acl
* more logs
* CORS Test Fix
* fix test_bucket_list_return_data
* default to private
* fix test_bucket_list_delimiter_not_skip_special
* default no acl
* add debug logging
* more logs
* use basic http client
remove logs also
* fixes
* debug
* Update stats.go
* debugging
* fix anonymous test expectation
anonymous user can read, as configured in s3 json.
5 months ago
632029fd8b
chore(deps): bump github.com/a-h/templ from 0.3.906 to 0.3.920 ( #7022 )
---
updated-dependencies:
- dependency-name: github.com/a-h/templ
dependency-version: 0.3.920
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
5 months ago
b3d8ff05b7
chore(deps): bump github.com/aws/aws-sdk-go-v2/config from 1.29.17 to 1.29.18 ( #7019 )
chore(deps): bump github.com/aws/aws-sdk-go-v2/config
Bumps [github.com/aws/aws-sdk-go-v2/config](https://github.com/aws/aws-sdk-go-v2 ) from 1.29.17 to 1.29.18.
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases )
- [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/main/changelog-template.json )
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/config/v1.29.17...config/v1.29.18 )
---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go-v2/config
dependency-version: 1.29.18
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
5 months ago
fd94a026ac
chore(deps): bump actions/setup-python from 4 to 5 ( #7021 )
Bumps [actions/setup-python](https://github.com/actions/setup-python ) from 4 to 5.
- [Release notes](https://github.com/actions/setup-python/releases )
- [Commits](https://github.com/actions/setup-python/compare/v4...v5 )
---
updated-dependencies:
- dependency-name: actions/setup-python
dependency-version: '5'
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
5 months ago
03b6b83419
chore(deps): bump github.com/klauspost/reedsolomon from 1.12.4 to 1.12.5 ( #7018 )
---
updated-dependencies:
- dependency-name: github.com/klauspost/reedsolomon
dependency-version: 1.12.5
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
5 months ago
325d452da6
chore(deps): bump gocloud.dev/pubsub/rabbitpubsub from 0.42.0 to 0.43.0 ( #7016 )
---
updated-dependencies:
- dependency-name: gocloud.dev/pubsub/rabbitpubsub
dependency-version: 0.43.0
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
5 months ago
289cba0e78
chore(deps): bump google.golang.org/api from 0.241.0 to 0.242.0 ( #7009 )
Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client ) from 0.241.0 to 0.242.0.
- [Release notes](https://github.com/googleapis/google-api-go-client/releases )
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md )
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.241.0...v0.242.0 )
---
updated-dependencies:
- dependency-name: google.golang.org/api
dependency-version: 0.242.0
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
5 months ago
3ba49871db
chore(deps): bump github.com/ydb-platform/ydb-go-sdk/v3 from 3.112.0 to 3.113.1 ( #7010 )
chore(deps): bump github.com/ydb-platform/ydb-go-sdk/v3
Bumps [github.com/ydb-platform/ydb-go-sdk/v3](https://github.com/ydb-platform/ydb-go-sdk ) from 3.112.0 to 3.113.1.
- [Release notes](https://github.com/ydb-platform/ydb-go-sdk/releases )
- [Changelog](https://github.com/ydb-platform/ydb-go-sdk/blob/master/CHANGELOG.md )
- [Commits](https://github.com/ydb-platform/ydb-go-sdk/compare/v3.112.0...v3.113.1 )
---
updated-dependencies:
- dependency-name: github.com/ydb-platform/ydb-go-sdk/v3
dependency-version: 3.113.1
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
5 months ago
b5bef082e0
chore(deps): bump github.com/aws/aws-sdk-go-v2 from 1.36.5 to 1.36.6 ( #7011 )
Bumps [github.com/aws/aws-sdk-go-v2](https://github.com/aws/aws-sdk-go-v2 ) from 1.36.5 to 1.36.6.
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases )
- [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/main/changelog-template.json )
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/v1.36.5...v1.36.6 )
---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go-v2
dependency-version: 1.36.6
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
5 months ago
3455fffacf
chore(deps): bump github.com/golang-jwt/jwt/v5 from 5.2.2 to 5.2.3 ( #7013 )
Bumps [github.com/golang-jwt/jwt/v5](https://github.com/golang-jwt/jwt ) from 5.2.2 to 5.2.3.
- [Release notes](https://github.com/golang-jwt/jwt/releases )
- [Changelog](https://github.com/golang-jwt/jwt/blob/main/VERSION_HISTORY.md )
- [Commits](https://github.com/golang-jwt/jwt/compare/v5.2.2...v5.2.3 )
---
updated-dependencies:
- dependency-name: github.com/golang-jwt/jwt/v5
dependency-version: 5.2.3
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
5 months ago
079adbfbae
chore(deps): bump github.com/aws/aws-sdk-go-v2/service/s3 from 1.83.0 to 1.84.1 ( #7014 )
chore(deps): bump github.com/aws/aws-sdk-go-v2/service/s3
Bumps [github.com/aws/aws-sdk-go-v2/service/s3](https://github.com/aws/aws-sdk-go-v2 ) from 1.83.0 to 1.84.1.
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases )
- [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/main/changelog-template.json )
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/service/s3/v1.83.0...service/s3/v1.84.1 )
---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go-v2/service/s3
dependency-version: 1.84.1
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
5 months ago
3a5ee18265
Fix versioning list only ( #7015 )
* fix listing objects
* address comments
* Update weed/s3api/s3api_object_versioning.go
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
* Update test/s3/versioning/s3_directory_versioning_test.go
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
---------
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
5 months ago
c196d03951
fix listing object versions ( #7006 )
* fix listing object versions
* Update s3api_object_versioning.go
* Update s3_directory_versioning_test.go
* check previous skipped tests
* fix test_versioning_stack_delete_merkers
* address test_bucket_list_return_data_versioning
* Update s3_directory_versioning_test.go
* fix test_versioning_concurrent_multi_object_delete
* fix test_versioning_obj_suspend_versions test
* fix empty owner
* fix listing versioned objects
* default owner
* fix path
5 months ago
bfe68984d5
fix logging
5 months ago
377f1f24c7
add basic object ACL ( #7004 )
* add back tests
* get put object acl
* check permission to put object acl
* rename file
* object list versions now contains owners
* set object owner
* refactoring
* Revert "add back tests"
This reverts commit 9adc507c45
5 months ago
85036936d1
Read write directory object ( #7003 )
* read directory object
* address comments
* address comments
* name should not have "/" prefix
* fix compilation
* refactor
5 months ago
41b5bac063
read directory object ( #7002 )
* read directory object
* address comments
* address comments
5 months ago
chrislu
Mohamed Sekour
Chris Lu
dependabot[bot]