Chris Lu
|
ff271aba16
|
Migrate from deprecated azure-storage-blob-go to modern Azure SDK
Migrates Azure Blob Storage integration from the deprecated
github.com/Azure/azure-storage-blob-go to the modern
github.com/Azure/azure-sdk-for-go/sdk/storage/azblob SDK.
## Changes
### Removed Files
- weed/remote_storage/azure/azure_highlevel.go
- Custom upload helper no longer needed with new SDK
### Updated Files
- weed/remote_storage/azure/azure_storage_client.go
- Migrated from ServiceURL/ContainerURL/BlobURL to Client-based API
- Updated client creation using NewClientWithSharedKeyCredential
- Replaced ListBlobsFlatSegment with NewListBlobsFlatPager
- Updated Download to DownloadStream with proper HTTPRange
- Replaced custom uploadReaderAtToBlockBlob with UploadStream
- Updated GetProperties, SetMetadata, Delete to use new client methods
- Fixed metadata conversion to return map[string]*string
- weed/replication/sink/azuresink/azure_sink.go
- Migrated from ContainerURL to Client-based API
- Updated client initialization
- Replaced AppendBlobURL with AppendBlobClient
- Updated error handling to use azcore.ResponseError
- Added streaming.NopCloser for AppendBlock
### New Test Files
- weed/remote_storage/azure/azure_storage_client_test.go
- Comprehensive unit tests for all client operations
- Tests for Traverse, ReadFile, WriteFile, UpdateMetadata, Delete
- Tests for metadata conversion function
- Benchmark tests
- Integration tests (skippable without credentials)
- weed/replication/sink/azuresink/azure_sink_test.go
- Unit tests for Azure sink operations
- Tests for CreateEntry, UpdateEntry, DeleteEntry
- Tests for cleanKey function
- Tests for configuration-based initialization
- Integration tests (skippable without credentials)
- Benchmark tests
### Dependency Updates
- go.mod: Removed github.com/Azure/azure-storage-blob-go v0.15.0
- go.mod: Made github.com/Azure/azure-sdk-for-go/sdk/storage/azblob v1.6.2 direct dependency
- All deprecated dependencies automatically cleaned up
## API Migration Summary
Old SDK → New SDK mappings:
- ServiceURL → Client (service-level operations)
- ContainerURL → ContainerClient
- BlobURL → BlobClient
- BlockBlobURL → BlockBlobClient
- AppendBlobURL → AppendBlobClient
- ListBlobsFlatSegment() → NewListBlobsFlatPager()
- Download() → DownloadStream()
- Upload() → UploadStream()
- Marker-based pagination → Pager-based pagination
- azblob.ResponseError → azcore.ResponseError
## Testing
All tests pass:
- ✅ Unit tests for metadata conversion
- ✅ Unit tests for helper functions (cleanKey)
- ✅ Interface implementation tests
- ✅ Build successful
- ✅ No compilation errors
- ✅ Integration tests available (require Azure credentials)
## Benefits
- ✅ Uses actively maintained SDK
- ✅ Better performance with modern API design
- ✅ Improved error handling
- ✅ Removes ~200 lines of custom upload code
- ✅ Reduces dependency count
- ✅ Better async/streaming support
- ✅ Future-proof against SDK deprecation
## Backward Compatibility
The changes are transparent to users:
- Same configuration parameters (account name, account key)
- Same functionality and behavior
- No changes to SeaweedFS API or user-facing features
- Existing Azure storage configurations continue to work
## Breaking Changes
None - this is an internal implementation change only.
|
4 weeks ago |
Aleksey Kosov
|
283d9e0079
|
Add context with request (#6824)
|
5 months ago |
|
chrislu
|
ec155022e7
|
"golang.org/x/exp/slices" => "slices" and go fmt
|
11 months ago |
|
chrislu
|
7a75928e7d
|
Azure sink: avoid overwriting existing files
|
1 year ago |
|
chrislu
|
70a4c98b00
|
refactor filer_pb.Entry and filer.Entry to use GetChunks()
for later locking on reading chunks
|
3 years ago |
|
chrislu
|
11f99836c3
|
filer.backup: backup small files if the file is saved in filer (saveToFilerLimit > 0)
fix https://github.com/seaweedfs/seaweedfs/issues/3468
|
3 years ago |
|
chrislu
|
26dbc6c905
|
move to https://github.com/seaweedfs/seaweedfs
|
3 years ago |
|
chrislu
|
bff1ccc1de
|
fix compilation
|
4 years ago |
|
Chris Lu
|
7c39a18ba5
|
update azure library
|
4 years ago |
|
Chris Lu
|
45bffc92a8
|
filer.backup: fix cloud sinks when updating entry
|
5 years ago |
|
Chris Lu
|
540441fd38
|
go fmt
|
5 years ago |
|
Chris Lu
|
678c54d705
|
data sink: add incremental mode
|
5 years ago |
|
Chris Lu
|
a8624c2e4f
|
read from alternative replica
related to https://github.com/chrislusf/seaweedfs/issues/1512
|
5 years ago |
|
Chris Lu
|
387ab6796f
|
filer: cross cluster synchronization
|
5 years ago |
|
Chris Lu
|
eb7929a971
|
rename filer2 to filer
|
5 years ago |
|
Chris Lu
|
c27e18aa6a
|
read possible old deleted chunks
|
5 years ago |
|
Chris Lu
|
c647deace1
|
file size support set file length
use Attr.FileSize and TotalChunkSize to determine file size
|
5 years ago |
|
Chris Lu
|
97d97f3528
|
go code can read and write chunk manifest
|
5 years ago |
|
Chris Lu
|
f282ed444b
|
refactoring
|
6 years ago |
|
Chris Lu
|
ae2ee379c0
|
consistent 64bit size
|
6 years ago |
|
Chris Lu
|
2e3f6ad3a9
|
filer: remember content is gzipped or not
|
6 years ago |
|
Chris Lu
|
13e215ee5c
|
filer: option to encrypt data on volume server
|
6 years ago |
|
Chris Lu
|
892e726eb9
|
avoid reusing context object
fix https://github.com/chrislusf/seaweedfs/issues/1182
|
6 years ago |
|
Chris Lu
|
d335f04de6
|
support env variables to overwrite toml file
|
6 years ago |
j.laycock
|
6fc6322c90
|
Change joeslay paths to chrislusf paths
|
6 years ago |
|
j.laycock
|
595a1beff0
|
Swap imports to use joeslay
|
6 years ago |
|
Chris Lu
|
b3b42bc947
|
replicate need to include new entry path
|
7 years ago |
Chris Lu
|
55bab1b456
|
add context.Context
|
7 years ago |
|
Chris Lu
|
8dfac6a4cf
|
working b2 sink
|
7 years ago |
|
Chris Lu
|
6a756136ef
|
go fmt
|
7 years ago |
|
Chris Lu
|
b861899919
|
use released version of azblob
replace https://github.com/chrislusf/seaweedfs/pull/754
|
7 years ago |
|
Chris Lu
|
08266b7256
|
go fmt
|
7 years ago |
|
Chris Lu
|
9f4c2f87d3
|
adding filer replication sink to Azure
|
7 years ago |
