You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
chrislu
ba2dcfc26c
refactor: make circuit breaker parameters configurable in FilerClient
The circuit breaker failure threshold (3) and reset timeout (30s) were
hardcoded, making it difficult to tune the client's behavior in different
deployment environments without modifying the code.
Problem:
func shouldSkipUnhealthyFiler(index int32) bool {
if failureCount < 3 { // Hardcoded threshold
return false
}
if time.Since(lastFailureTime) > 30*time.Second { // Hardcoded timeout
return false
}
}
Different environments have different needs:
- High-traffic production: may want lower threshold (2) for faster failover
- Development/testing: may want higher threshold (5) to tolerate flaky networks
- Low-latency services: may want shorter reset timeout (10s)
- Batch processing: may want longer reset timeout (60s)
Solution:
1. Added fields to FilerClientOption:
- FailureThreshold int32 (default: 3)
- ResetTimeout time.Duration (default: 30s)
2. Added fields to FilerClient:
- failureThreshold int32
- resetTimeout time.Duration
3. Applied defaults in NewFilerClient with option override:
failureThreshold := int32(3)
resetTimeout := 30 * time.Second
if opt.FailureThreshold > 0 {
failureThreshold = opt.FailureThreshold
}
if opt.ResetTimeout > 0 {
resetTimeout = opt.ResetTimeout
}
4. Updated shouldSkipUnhealthyFiler to use configurable values:
if failureCount < fc.failureThreshold { ... }
if time.Since(lastFailureTime) > fc.resetTimeout { ... }
Benefits:
✓ Tunable for different deployment environments
✓ Backward compatible (defaults match previous hardcoded values)
✓ No breaking changes to existing code
✓ Better maintainability and flexibility
Example usage:
// Aggressive failover for low-latency production
fc := wdclient.NewFilerClient(filers, dialOpt, dc, &wdclient.FilerClientOption{
FailureThreshold: 2,
ResetTimeout: 10 * time.Second,
})
// Tolerant of flaky networks in development
fc := wdclient.NewFilerClient(filers, dialOpt, dc, &wdclient.FilerClientOption{
FailureThreshold: 5,
ResetTimeout: 60 * time.Second,
})
|
2 weeks ago |
| .. |
|
admin
|
muted texts
|
1 month ago |
|
cluster
|
adds FilerClient to use cached volume id
|
2 weeks ago |
|
command
|
backup: handle volume not found when backing up (#7465)
|
3 weeks ago |
|
credential
|
Filer Store: postgres backend support pgbouncer (#7077)
|
4 months ago |
|
filer
|
refactor: address code review feedback on comments and style
|
2 weeks ago |
|
filer_client
|
Clean up logs and deprecated functions (#7339)
|
2 months ago |
|
glog
|
Add Kafka Gateway (#7231)
|
2 months ago |
|
iam
|
S3: Enforce bucket policy (#7471)
|
3 weeks ago |
|
iamapi
|
refactor: address code review feedback on comments and style
|
2 weeks ago |
|
images
|
Migrates from disintegration/imaging c2019 to cognusion/imaging c2024. (#5533)
|
2 years ago |
|
kms
|
S3 API: Add integration with KMS providers (#7152)
|
3 months ago |
|
mount
|
improve: address remaining code review findings
|
2 weeks ago |
|
mq
|
S3: Directly read write volume servers (#7481)
|
2 weeks ago |
|
notification
|
fix: dead letter message log message (#7072)
|
4 months ago |
|
operation
|
S3: Directly read write volume servers (#7481)
|
2 weeks ago |
|
pb
|
S3: Directly read write volume servers (#7481)
|
2 weeks ago |
|
query
|
Fix date string parsing bug for the SQL Engine. (#7446)
|
4 weeks ago |
|
remote_storage
|
Filer: Fixed critical bugs in the Azure SDK migration (PR #7310) (#7401)
|
1 month ago |
|
replication
|
Filer: Fixed critical bugs in the Azure SDK migration (PR #7310) (#7401)
|
1 month ago |
|
s3api
|
fix: FilerClient supports multiple filer addresses for high availability
|
2 weeks ago |
|
security
|
remove spoof-able request header (#7103)
|
4 months ago |
|
sequence
|
remove unused function
|
1 year ago |
|
server
|
refactor: remove unnecessary KeepMasterClientConnected wrapper in filer
|
2 weeks ago |
|
sftpd
|
S3 API: Advanced IAM System (#7160)
|
3 months ago |
|
shell
|
Account Info (#7507)
|
2 weeks ago |
|
static
|
Fix Broken Links (#5287)
|
2 years ago |
|
stats
|
[volume] refactor and add metrics for flight upload and download data limit condition (#6920)
|
5 months ago |
|
storage
|
Volume Server: avoid aggressive volume assignment (#7501)
|
2 weeks ago |
|
telemetry
|
convert error fromating to %w everywhere (#6995)
|
5 months ago |
|
topology
|
master: fix negative active volumes (#7440)
|
1 month ago |
|
util
|
S3: Directly read write volume servers (#7481)
|
2 weeks ago |
|
wdclient
|
refactor: make circuit breaker parameters configurable in FilerClient
|
2 weeks ago |
|
worker
|
go fmt
|
1 month ago |
|
Makefile
|
test versioning also (#7000)
|
5 months ago |
|
weed.go
|
set exit status
|
9 months ago |