You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
chrislu
ba2dcfc26c
refactor: make circuit breaker parameters configurable in FilerClient
The circuit breaker failure threshold (3) and reset timeout (30s) were
hardcoded, making it difficult to tune the client's behavior in different
deployment environments without modifying the code.
Problem:
func shouldSkipUnhealthyFiler(index int32) bool {
if failureCount < 3 { // Hardcoded threshold
return false
}
if time.Since(lastFailureTime) > 30*time.Second { // Hardcoded timeout
return false
}
}
Different environments have different needs:
- High-traffic production: may want lower threshold (2) for faster failover
- Development/testing: may want higher threshold (5) to tolerate flaky networks
- Low-latency services: may want shorter reset timeout (10s)
- Batch processing: may want longer reset timeout (60s)
Solution:
1. Added fields to FilerClientOption:
- FailureThreshold int32 (default: 3)
- ResetTimeout time.Duration (default: 30s)
2. Added fields to FilerClient:
- failureThreshold int32
- resetTimeout time.Duration
3. Applied defaults in NewFilerClient with option override:
failureThreshold := int32(3)
resetTimeout := 30 * time.Second
if opt.FailureThreshold > 0 {
failureThreshold = opt.FailureThreshold
}
if opt.ResetTimeout > 0 {
resetTimeout = opt.ResetTimeout
}
4. Updated shouldSkipUnhealthyFiler to use configurable values:
if failureCount < fc.failureThreshold { ... }
if time.Since(lastFailureTime) > fc.resetTimeout { ... }
Benefits:
✓ Tunable for different deployment environments
✓ Backward compatible (defaults match previous hardcoded values)
✓ No breaking changes to existing code
✓ Better maintainability and flexibility
Example usage:
// Aggressive failover for low-latency production
fc := wdclient.NewFilerClient(filers, dialOpt, dc, &wdclient.FilerClientOption{
FailureThreshold: 2,
ResetTimeout: 10 * time.Second,
})
// Tolerant of flaky networks in development
fc := wdclient.NewFilerClient(filers, dialOpt, dc, &wdclient.FilerClientOption{
FailureThreshold: 5,
ResetTimeout: 60 * time.Second,
})
|
2 months ago |
| .. |
|
admin
|
muted texts
|
2 months ago |
|
cluster
|
adds FilerClient to use cached volume id
|
2 months ago |
|
command
|
backup: handle volume not found when backing up (#7465)
|
2 months ago |
|
credential
|
Filer Store: postgres backend support pgbouncer (#7077)
|
5 months ago |
|
filer
|
refactor: address code review feedback on comments and style
|
2 months ago |
|
filer_client
|
Clean up logs and deprecated functions (#7339)
|
3 months ago |
|
glog
|
Add Kafka Gateway (#7231)
|
3 months ago |
|
iam
|
S3: Enforce bucket policy (#7471)
|
2 months ago |
|
iamapi
|
refactor: address code review feedback on comments and style
|
2 months ago |
|
images
|
Migrates from disintegration/imaging c2019 to cognusion/imaging c2024. (#5533)
|
2 years ago |
|
kms
|
S3 API: Add integration with KMS providers (#7152)
|
5 months ago |
|
mount
|
improve: address remaining code review findings
|
2 months ago |
|
mq
|
S3: Directly read write volume servers (#7481)
|
2 months ago |
|
notification
|
fix: dead letter message log message (#7072)
|
5 months ago |
|
operation
|
S3: Directly read write volume servers (#7481)
|
2 months ago |
|
pb
|
S3: Directly read write volume servers (#7481)
|
2 months ago |
|
query
|
Fix date string parsing bug for the SQL Engine. (#7446)
|
2 months ago |
|
remote_storage
|
Filer: Fixed critical bugs in the Azure SDK migration (PR #7310) (#7401)
|
3 months ago |
|
replication
|
Filer: Fixed critical bugs in the Azure SDK migration (PR #7310) (#7401)
|
3 months ago |
|
s3api
|
fix: FilerClient supports multiple filer addresses for high availability
|
2 months ago |
|
security
|
remove spoof-able request header (#7103)
|
5 months ago |
|
sequence
|
remove unused function
|
2 years ago |
|
server
|
refactor: remove unnecessary KeepMasterClientConnected wrapper in filer
|
2 months ago |
|
sftpd
|
S3 API: Advanced IAM System (#7160)
|
4 months ago |
|
shell
|
Account Info (#7507)
|
2 months ago |
|
static
|
Fix Broken Links (#5287)
|
2 years ago |
|
stats
|
[volume] refactor and add metrics for flight upload and download data limit condition (#6920)
|
6 months ago |
|
storage
|
Volume Server: avoid aggressive volume assignment (#7501)
|
2 months ago |
|
telemetry
|
convert error fromating to %w everywhere (#6995)
|
6 months ago |
|
topology
|
master: fix negative active volumes (#7440)
|
2 months ago |
|
util
|
S3: Directly read write volume servers (#7481)
|
2 months ago |
|
wdclient
|
refactor: make circuit breaker parameters configurable in FilerClient
|
2 months ago |
|
worker
|
go fmt
|
3 months ago |
|
Makefile
|
test versioning also (#7000)
|
6 months ago |
|
weed.go
|
set exit status
|
10 months ago |