7e6db15a88
docs(volume-server): record slash fid-path native validation progress
4 weeks ago
b010af88fb
docs(volume-server): log native write prevalidation parity progress
4 weeks ago
47156eb8ce
docs(volume-server): record native malformed-fid parity milestone
4 weeks ago
120aa956a0
docs(volume-server): update native parity progress logs
4 weeks ago
532bf863ad
docs(volume-server): log native ui static method parity progress
4 weeks ago
b0447d2479
docs(volume-server): track native rust options parity milestone
4 weeks ago
d2a6066181
docs(volume-server): log native rust status healthz milestone
4 weeks ago
c7c7be42ed
docs(volume-server): track native launcher bootstrap progress
4 weeks ago
14c863dbff
docs(volume-server): refocus plan on native rust parity
4 weeks ago
6bb9d8bac2
docs(volume_server): log head readDeleted parity coverage
4 weeks ago
cc80ad3643
test(volume_server/http): add head readDeleted parity coverage
4 weeks ago
9009e38f7b
docs(volume_server): log ping volume-server unreachable coverage
4 weeks ago
b9fbb85af2
test(volume_server/grpc): add ping unreachable volume-server target case
4 weeks ago
47d3001572
docs(volume_server): log csv query payload parity coverage
4 weeks ago
a12dd5f8d3
test(volume_server/grpc): cover csv-query payload no-output parity
4 weeks ago
8e614486a3
docs(volume_server): log tail-receiver interruption coverage
4 weeks ago
a5864c3eb6
test(volume_server/grpc): cover tail-receiver source-unavailable branch
4 weeks ago
6302809442
docs(volume_server): log tail sender cancellation coverage
4 weeks ago
27a80f7607
test(volume_server/grpc): add tail-sender cancellation interruption coverage
4 weeks ago
ec429e0361
docs(volume_server): log framework port-range hardening and rerun
4 weeks ago
90e82b15ce
test(volume_server/framework): allocate volume ports within safe grpc-offset range
4 weeks ago
a3e1ee1653
docs(volume_server): log mkcol method parity coverage
4 weeks ago
2ab30900d4
test(volume_server/http): add mkcol unsupported-method parity
4 weeks ago
62ee14fa61
docs(volume_server): log read-all-needles multi-volume coverage
4 weeks ago
ab95a6ef15
test(volume_server/grpc): cover read-all-needles multi-volume success
4 weeks ago
24965fd489
docs(volume_server): log head conditional precedence coverage
4 weeks ago
ed23e290fc
test(volume_server/http): expand head conditional precedence coverage
4 weeks ago
9b57fb6961
docs(volume_server): log ec batch delete success coverage
4 weeks ago
1bb40b6bc5
test(volume_server/grpc): add ec batch delete success coverage
4 weeks ago
34e342da63
docs(volume_server): log replicated write failure coverage
4 weeks ago
4835d34438
test(volume_server/http): cover replicated write failure when replication unmet
4 weeks ago
5814729def
docs(volume_server): log ec-only read meta coverage
4 weeks ago
37bf9b5ebf
test(volume_server/grpc): cover ec-only read needle meta unsupported path
4 weeks ago
19201df6d7
docs(volume_server): log oversized upload limit coverage
4 weeks ago
4d61cbdeed
test(volume_server/http): cover oversized upload file-size limit rejection
4 weeks ago
3ce883624e
docs(volume_server): log jwt ui access override coverage
4 weeks ago
de974c05d5
test(volume_server/http): cover jwt ui access override behavior
4 weeks ago
7768fda023
docs(volume_server): record proxy-mode validation and CI matrix
4 weeks ago
6ce4d7eded
docs(volume_server): record rust-mode full-suite validation
4 weeks ago
d402573ea8
docs(volume_server): document rust-mode harness and tracking
4 weeks ago
7beab85c21
test(volume_server/framework): support selectable volume server binary
4 weeks ago
f44e25b422
fix(iam): ensure access key status is persisted and defaulted to Active ( #8341 )
* Fix master leader election startup issue
Fixes #error-log-leader-not-selected-yet
* not useful test
* fix(iam): ensure access key status is persisted and defaulted to Active
* make pb
* update tests
* using constants
1 month ago
35b6e895cc
build(deps): bump org.apache.avro:avro from 1.11.4 to 1.11.5 in /test/kafka/kafka-client-loadtest/tools ( #8339 )
build(deps): bump org.apache.avro:avro
Bumps org.apache.avro:avro from 1.11.4 to 1.11.5.
---
updated-dependencies:
- dependency-name: org.apache.avro:avro
dependency-version: 1.11.5
dependency-type: direct:production
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
1 month ago
49a64f50f1
Add session policy support to IAM ( #8338 )
* Add session policy support to IAM
- Implement policy evaluation for session tokens in policy_engine.go
- Add session_policy field to session claims for tracking applied policies
- Update STS service to include session policies in token generation
- Add IAM integration tests for session policy validation
- Update IAM manager to support policy attachment to sessions
- Extend S3 API STS endpoint to handle session policy restrictions
* fix: optimize session policy evaluation and add documentation
* sts: add NormalizeSessionPolicy helper for inline session policies
* sts: support inline session policies for AssumeRoleWithWebIdentity and credential-based flows
* s3api: parse and normalize Policy parameter for STS HTTP handlers
* tests: add session policy unit tests and integration tests for inline policy downscoping
* tests: add s3tables STS inline policy integration
* iam: handle user principals and validate tokens
* sts: enforce inline session policy size limit
* tests: harden s3tables STS integration config
* iam: clarify principal policy resolution errors
* tests: improve STS integration endpoint selection
1 month ago
beeb375a88
Add volume server integration test suite and CI workflow ( #8322 )
* docs(volume_server): add integration test development plan
* test(volume_server): add integration harness and profile matrix
* test(volume_server/http): add admin and options integration coverage
* test(volume_server/grpc): add state and status integration coverage
* test(volume_server): auto-build weed binary and harden cluster startup
* test(volume_server/http): add upload read range head delete coverage
* test(volume_server/grpc): expand admin lifecycle and state coverage
* docs(volume_server): update progress tracker for implemented tests
* test(volume_server/http): cover if-none-match and invalid-range branches
* test(volume_server/grpc): add batch delete integration coverage
* docs(volume_server): log latest HTTP and gRPC test coverage
* ci(volume_server): run volume server integration tests in github actions
* test(volume_server/grpc): add needle status configure ping and leave coverage
* docs(volume_server): record additional grpc coverage progress
* test(volume_server/grpc): add vacuum integration coverage
* docs(volume_server): record vacuum test coverage progress
* test(volume_server/grpc): add read and write needle blob error-path coverage
* docs(volume_server): record data rw grpc coverage progress
* test(volume_server/http): add jwt auth integration coverage
* test(volume_server/grpc): add sync copy and stream error-path coverage
* docs(volume_server): record jwt and sync/copy test coverage
* test(volume_server/grpc): add scrub and query integration coverage
* test(volume_server/grpc): add volume tail sender and receiver coverage
* docs(volume_server): record scrub query and tail test progress
* test(volume_server/grpc): add readonly writable and collection lifecycle coverage
* test(volume_server/http): add public-port cors and method parity coverage
* test(volume_server/grpc): add blob meta and read-all success path coverage
* test(volume_server/grpc): expand scrub and query variation coverage
* test(volume_server/grpc): add tiering and remote fetch error-path coverage
* test(volume_server/http): add unchanged write and delete edge-case coverage
* test(volume_server/grpc): add ping unknown and unreachable target coverage
* test(volume_server/grpc): add volume delete only-empty variation coverage
* test(volume_server/http): add jwt fid-mismatch auth coverage
* test(volume_server/grpc): add scrub ec auto-select empty coverage
* test(volume_server/grpc): stabilize ping timestamp assertion
* docs(volume_server): update integration coverage progress log
* test(volume_server/grpc): add tier remote backend and config variation coverage
* docs(volume_server): record tier remote variation progress
* test(volume_server/grpc): add incremental copy and receive-file protocol coverage
* test(volume_server/http): add read path shape and if-modified-since coverage
* test(volume_server/grpc): add copy-file compaction and receive-file success coverage
* test(volume_server/http): add passthrough headers and static asset coverage
* test(volume_server/grpc): add ping filer unreachable coverage
* docs(volume_server): record copy receive and http variant progress
* test(volume_server/grpc): add erasure coding maintenance and missing-path coverage
* docs(volume_server): record initial erasure coding rpc coverage
* test(volume_server/http): add multi-range multipart response coverage
* docs(volume_server): record multi-range http coverage progress
* test(volume_server/grpc): add query empty-stripe no-match coverage
* docs(volume_server): record query no-match stream behavior coverage
* test(volume_server/http): add upload throttling timeout and replicate bypass coverage
* docs(volume_server): record upload throttling coverage progress
* test(volume_server/http): add download throttling timeout coverage
* docs(volume_server): record download throttling coverage progress
* test(volume_server/http): add jwt wrong-cookie fid mismatch coverage
* docs(volume_server): record jwt wrong-cookie mismatch coverage
* test(volume_server/http): add jwt expired-token rejection coverage
* docs(volume_server): record jwt expired-token coverage
* test(volume_server/http): add jwt query and cookie transport coverage
* docs(volume_server): record jwt token transport coverage
* test(volume_server/http): add jwt token-source precedence coverage
* docs(volume_server): record jwt token-source precedence coverage
* test(volume_server/http): add jwt header-over-cookie precedence coverage
* docs(volume_server): record jwt header cookie precedence coverage
* test(volume_server/http): add jwt query-over-cookie precedence coverage
* docs(volume_server): record jwt query cookie precedence coverage
* test(volume_server/grpc): add setstate version mismatch and nil-state coverage
* docs(volume_server): record setstate validation coverage
* test(volume_server/grpc): add readonly persist-true lifecycle coverage
* docs(volume_server): record readonly persist variation coverage
* test(volume_server/http): add options origin cors header coverage
* docs(volume_server): record options origin cors coverage
* test(volume_server/http): add trace unsupported-method parity coverage
* docs(volume_server): record trace method parity coverage
* test(volume_server/grpc): add batch delete cookie-check variation coverage
* docs(volume_server): record batch delete cookie-check coverage
* test(volume_server/grpc): add admin lifecycle missing and maintenance variants
* docs(volume_server): record admin lifecycle edge-case coverage
* test(volume_server/grpc): add mixed batch delete status matrix coverage
* docs(volume_server): record mixed batch delete matrix coverage
* test(volume_server/http): add jwt-profile ui access gating coverage
* docs(volume_server): record jwt ui-gating http coverage
* test(volume_server/http): add propfind unsupported-method parity coverage
* docs(volume_server): record propfind method parity coverage
* test(volume_server/grpc): add volume configure success and rollback-path coverage
* docs(volume_server): record volume configure branch coverage
* test(volume_server/grpc): add volume needle status missing-path coverage
* docs(volume_server): record volume needle status error-path coverage
* test(volume_server/http): add readDeleted query behavior coverage
* docs(volume_server): record readDeleted http behavior coverage
* test(volume_server/http): add delete ts override parity coverage
* docs(volume_server): record delete ts parity coverage
* test(volume_server/grpc): add invalid blob/meta offset coverage
* docs(volume_server): record invalid blob/meta offset coverage
* test(volume_server/grpc): add read-all mixed volume abort coverage
* docs(volume_server): record read-all mixed-volume abort coverage
* test(volume_server/http): assert head response body parity
* docs(volume_server): record head body parity assertion
* test(volume_server/grpc): assert status state and memory payload completeness
* docs(volume_server): record volume server status payload coverage
* test(volume_server/grpc): add batch delete chunk-manifest rejection coverage
* docs(volume_server): record batch delete chunk-manifest coverage
* test(volume_server/grpc): add query cookie-mismatch eof parity coverage
* docs(volume_server): record query cookie-mismatch parity coverage
* test(volume_server/grpc): add ping master success target coverage
* docs(volume_server): record ping master success coverage
* test(volume_server/http): add head if-none-match conditional parity
* docs(volume_server): record head if-none-match parity coverage
* test(volume_server/http): add head if-modified-since parity coverage
* docs(volume_server): record head if-modified-since parity coverage
* test(volume_server/http): add connect unsupported-method parity coverage
* docs(volume_server): record connect method parity coverage
* test(volume_server/http): assert options allow-headers cors parity
* docs(volume_server): record options allow-headers coverage
* test(volume_server/framework): add dual volume cluster integration harness
* test(volume_server/http): add missing-local read mode proxy redirect local coverage
* docs(volume_server): record read mode missing-local matrix coverage
* test(volume_server/http): add download over-limit replica proxy fallback coverage
* docs(volume_server): record download replica fallback coverage
* test(volume_server/http): add missing-local readDeleted proxy redirect parity coverage
* docs(volume_server): record missing-local readDeleted mode coverage
* test(volume_server/framework): add single-volume cluster with filer harness
* test(volume_server/grpc): add ping filer success target coverage
* docs(volume_server): record ping filer success coverage
* test(volume_server/http): add proxied-loop guard download timeout coverage
* docs(volume_server): record proxied-loop download coverage
* test(volume_server/http): add disabled upload and download limit coverage
* docs(volume_server): record disabled throttling path coverage
* test(volume_server/grpc): add idempotent volume server leave coverage
* docs(volume_server): record leave idempotence coverage
* test(volume_server/http): add redirect collection query preservation coverage
* docs(volume_server): record redirect collection query coverage
* test(volume_server/http): assert admin server headers on status and health
* docs(volume_server): record admin server header coverage
* test(volume_server/http): assert healthz request-id echo parity
* docs(volume_server): record healthz request-id parity coverage
* test(volume_server/http): add over-limit invalid-vid download branch coverage
* docs(volume_server): record over-limit invalid-vid branch coverage
* test(volume_server/http): add public-port static asset coverage
* docs(volume_server): record public static endpoint coverage
* test(volume_server/http): add public head method parity coverage
* docs(volume_server): record public head parity coverage
* test(volume_server/http): add throttling wait-then-proceed path coverage
* docs(volume_server): record throttling wait-then-proceed coverage
* test(volume_server/http): add read cookie-mismatch not-found coverage
* docs(volume_server): record read cookie-mismatch coverage
* test(volume_server/http): add throttling timeout-recovery coverage
* docs(volume_server): record throttling timeout-recovery coverage
* test(volume_server/grpc): add ec generate mount info unmount lifecycle coverage
* docs(volume_server): record ec positive lifecycle coverage
* test(volume_server/grpc): add ec shard read and blob delete lifecycle coverage
* docs(volume_server): record ec shard read/blob delete lifecycle coverage
* test(volume_server/grpc): add ec rebuild and to-volume error branch coverage
* docs(volume_server): record ec rebuild and to-volume branch coverage
* test(volume_server/grpc): add ec shards-to-volume success roundtrip coverage
* docs(volume_server): record ec shards-to-volume success coverage
* test(volume_server/grpc): add ec receive and copy-file missing-source coverage
* docs(volume_server): record ec receive and copy-file coverage
* test(volume_server/grpc): add ec last-shard delete cleanup coverage
* docs(volume_server): record ec last-shard delete cleanup coverage
* test(volume_server/grpc): add volume copy success path coverage
* docs(volume_server): record volume copy success coverage
* test(volume_server/grpc): add volume copy overwrite-destination coverage
* docs(volume_server): record volume copy overwrite coverage
* test(volume_server/http): add write error-path variant coverage
* docs(volume_server): record http write error-path coverage
* test(volume_server/http): add conditional header precedence coverage
* docs(volume_server): record conditional header precedence coverage
* test(volume_server/http): add oversized combined range guard coverage
* docs(volume_server): record oversized range guard coverage
* test(volume_server/http): add image resize and crop read coverage
* docs(volume_server): record image transform coverage
* test(volume_server/http): add chunk-manifest expansion and bypass coverage
* docs(volume_server): record chunk-manifest read coverage
* test(volume_server/http): add compressed read encoding matrix coverage
* docs(volume_server): record compressed read matrix coverage
* test(volume_server/grpc): add tail receiver source replication coverage
* docs(volume_server): record tail receiver replication coverage
* test(volume_server/grpc): add tail sender large-needle chunking coverage
* docs(volume_server): record tail sender chunking coverage
* test(volume_server/grpc): add ec-backed volume needle status coverage
* docs(volume_server): record ec-backed needle status coverage
* test(volume_server/grpc): add ec shard copy from peer success coverage
* docs(volume_server): record ec shard copy success coverage
* test(volume_server/http): add chunk-manifest delete child cleanup coverage
* docs(volume_server): record chunk-manifest delete cleanup coverage
* test(volume_server/http): add chunk-manifest delete failure-path coverage
* docs(volume_server): record chunk-manifest delete failure coverage
* test(volume_server/grpc): add ec shard copy source-unavailable coverage
* docs(volume_server): record ec shard copy source-unavailable coverage
* parallel
1 month ago
796f23f68a
Fix STS InvalidAccessKeyId and request body consumption issues ( #8328 )
* Fix STS InvalidAccessKeyId and request body consumption in Lakekeeper integration test
* Remove debug prints
* Add Lakekeeper integration tests to CI
* Fix connection refused in CI by binding to 0.0.0.0
* Add timeout to docker run in Lakekeeper integration test
* Update weed/s3api/auth_credentials.go
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
1 month ago
25ea48227f
Fix STS temporary credentials to use ASIA prefix instead of AKIA ( #8326 )
Temporary credentials from STS AssumeRole were using "AKIA" prefix
(permanent IAM user credentials) instead of "ASIA" prefix (temporary
security credentials). This violates AWS conventions and may cause
compatibility issues with AWS SDKs that validate credential types.
Changes:
- Rename generateAccessKeyId to generateTemporaryAccessKeyId for clarity
- Update function to use ASIA prefix for temporary credentials
- Add unit tests to verify ASIA prefix format (weed/iam/sts/credential_prefix_test.go)
- Add integration test to verify ASIA prefix in S3 API (test/s3/iam/s3_sts_credential_prefix_test.go)
- Ensure AWS-compatible credential format (ASIA + 16 hex chars)
The credentials are already deterministic (SHA256-based from session ID)
and the SessionToken is correctly set to the JWT token, so this is just
a prefix fix to follow AWS standards.
Fixes #8312
1 month ago
0082c47e04
Test: Add RisingWave DML verification test ( #8317 )
* Test: Verify RisingWave DML operations (INSERT, UPDATE, DELETE) support
* Test: Refine RisingWave DML test (remove sleeps, use polling)
1 month ago
c1a9263e37
Fix STS AssumeRole with POST body param ( #8320 )
* Fix STS AssumeRole with POST body param and add integration test
* Add STS integration test to CI workflow
* Address code review feedback: fix HPP vulnerability and style issues
* Refactor: address code review feedback
- Fix HTTP Parameter Pollution vulnerability in UnifiedPostHandler
- Refactor permission check logic for better readability
- Extract test helpers to testutil/docker.go to reduce duplication
- Clean up imports and simplify context setting
* Add SigV4-style test variant for AssumeRole POST body routing
- Added ActionInBodyWithSigV4Style test case to validate real-world scenario
- Test confirms routing works correctly for AWS SigV4-signed requests
- Addresses code review feedback about testing with SigV4 signatures
* Fix: always set identity in context when non-nil
- Ensure UnifiedPostHandler always calls SetIdentityInContext when identity is non-nil
- Only call SetIdentityNameInContext when identity.Name is non-empty
- This ensures downstream handlers (embeddedIam.DoActions) always have access to identity
- Addresses potential issue where empty identity.Name would skip context setting
1 month ago
b8ef48c8f1
Add RisingWave catalog tests ( #8308 )
* Add RisingWave catalog tests for S3 tables
* Add RisingWave catalog integration tests to CI workflow
* Refactor RisingWave catalog tests based on PR feedback
* Address PR feedback: optimize checks, cleanup logs
* fix tests
* consistent
1 month ago
Chris Lu
Chris Lu
dependabot[bot]