5c1e6e904d
CRITICAL: Restore task conflict definitions to prevent data integrity issues
- Restore conflicts between vacuum, balance, erasure_coding, and ec_vacuum tasks
- Prevent dangerous concurrent operations on same volumes/resources
- Add comprehensive task conflict matrix to avoid race conditions
- This addresses a serious safety regression where all conflicts were removed
Critical conflicts restored:
- vacuum ↔ balance, erasure_coding, ec_vacuum
- balance ↔ vacuum, erasure_coding, ec_vacuum
- erasure_coding ↔ vacuum, balance, ec_vacuum
- ec_vacuum ↔ vacuum, balance, erasure_coding
- replication ↔ vacuum, balance (destructive ops)
5 months ago
04e3a68b61
Replace deprecated ioutil with os package functions
- Replace ioutil.WriteFile with os.WriteFile
- Replace ioutil.ReadFile with os.ReadFile
- Remove deprecated io/ioutil import
- Aligns with Go 1.16+ best practices and modern Go standards
5 months ago
c220ad1e69
Replace bubble sort with idiomatic sort.Slice in EC shard management
- Replace O(n²) bubble sort implementation with efficient sort.Slice
- More concise, readable, and performant for larger slices
- Uses idiomatic Go sorting pattern
5 months ago
c9ca213d4c
Fix linter error in master_grpc_ec_generation_test.go
Remove orphaned code that was causing syntax error
5 months ago
4ec743583d
Address PR #7116 review comments
- Fix CodeQL security issue: Add bounds checking for int64 to uint8 conversion in disk_location_ec.go
- Replace goto with idiomatic map approach in ec_shard_management.go
- Fix EC volume handling in maintenance_scanner.go: add support for EC-only volumes
- Fix test failures in master_grpc_ec_generation_test.go: handle raft leadership issues
5 months ago
5bb475c572
Fixed CodeQL Security Issue
5 months ago
cd75202da8
Replaced goto with Idiomatic Map Approach
5 months ago
cdba261913
add safety check
5 months ago
179bcb777e
Delete ec_vacuum_integration_test.go
5 months ago
802267b281
validates all critical upgrade scenarios for the generation-aware EC vacuum system
5 months ago
d6e335d6e2
Integration test: end-to-end vacuum G→G+1 with live reads (zero downtime)
5 months ago
884da0496c
testing mixed generation
5 months ago
60f975aa20
add unit tests
5 months ago
a3d0509c92
logs
5 months ago
2e51e1dab2
ec volume UI rendering version
5 months ago
a187f103d1
normal volume CompactionRevision
5 months ago
3087da07db
metrics with generation
5 months ago
3ef8a9f3b2
Mixed-version cluster compatibility
5 months ago
d326affc4f
default generation
5 months ago
d341db8861
Update ec_vacuum_task.go
5 months ago
cf5f0b8e74
cleanupGracePeriod
5 months ago
0bcb64140f
Add activation step to EC vacuum: call ActivateEcGeneration when G+1 ready
5 months ago
62d89fa60b
EC vacuum distribution updated for generation-aware mount/copy RPCs
5 months ago
de9399761b
EC vacuum task updated for generation-aware operation
5 months ago
8797e73523
cachedLookupEcShardLocations updated for generation-specific caching
5 months ago
f00dc46607
VolumeEcShardRead to read from correct (vid, generation) EcVolume
5 months ago
1b41544f97
detecting ec volumes
5 months ago
e9033136aa
todo
5 months ago
56d1ed77c1
copy file
5 months ago
9e2e600b6d
VolumeEcShardsGenerate updated for generation-specific file creation
5 months ago
99f132729c
MountEcShards/UnmountEcShards updated for generation support
5 months ago
8c31d5e331
EcVolume creation properly refactored
5 months ago
ef5f9f629a
Generation file layout
5 months ago
50cc17e8fa
ActivateEcGeneration RPC implemented
5 months ago
870b2ffe07
LookupEcVolume handle generation parameter
5 months ago
e4f266d927
Active generation tracking implemented
5 months ago
5650b6f505
Add master_pb.ActivateEcGenerationRequest/Response RPC definition
5 months ago
09d740db47
Add `generation` field to volume_server_pb.VolumeEcShardReadRequest
5 months ago
8c54b72af3
Add `generation` field to volume_server_pb VolumeEcShards{Copy,Generate,Rebuild,ToVolume}Request
5 months ago
312fee4ddf
Add `generation` field to volume_server_pb VolumeEcShards{Mount,Unmount}Request
5 months ago
03ad8f2490
Add `generation` field to master_pb.VolumeEcShardInformationMessage
5 months ago
fc666e2e48
collect ec volume deleted bytes
5 months ago
3a9ec6f30c
Update worker.go
5 months ago
8603a3a1e9
ec vacuum task need to register itself
5 months ago
bea9ca3616
generate ec data and deleted data
5 months ago
32e7ba2f9d
adding ec vacuum
5 months ago
05a0cc156b
Self-Contained Design
To prove the system is truly self-contained:
To add a new task:
Create a task package (e.g., worker/tasks/compression/)
Import it: _ "github.com/.../worker/tasks/compression"
That's it! No other changes needed.
To remove a task:
Delete the task package directory
Remove the import line
That's it! No other changes needed.
5 months ago
96d6d27607
remove ❌ Vacuum - Completely removed ❌ Balance - Completely removed
5 months ago
c7f291e4e4
simplify logging
5 months ago
25bbf4c3d4
Admin UI: Fetch task logs ( #7114 )
* show task details
* loading tasks
* task UI works
* generic rendering
* rendering the export link
* removing placementConflicts from task parameters
* remove TaskSourceLocation
* remove "Server ID" column
* rendering balance task source
* sources and targets
* fix ec task generation
* move info
* render timeline
* simplified worker id
* simplify
* read task logs from worker
* isValidTaskID
* address comments
* Update weed/worker/tasks/balance/execution.go
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Update weed/worker/tasks/erasure_coding/ec_task.go
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Update weed/worker/tasks/task_log_handler.go
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* fix shard ids
* plan distributing shard id
* rendering planned shards in task details
* remove Conflicts
* worker logs correctly
* pass in dc and rack
* task logging
* Update weed/admin/maintenance/maintenance_queue.go
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
* display log details
* logs have fields now
* sort field keys
* fix link
* fix collection filtering
* avoid hard coded ec shard counts
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
5 months ago
chrislu
Chris Lu