diff --git a/k8s/charts/seaweedfs/README.md b/k8s/charts/seaweedfs/README.md index 30885aee3..7f27cb22e 100644 --- a/k8s/charts/seaweedfs/README.md +++ b/k8s/charts/seaweedfs/README.md @@ -145,6 +145,191 @@ stringData: seaweedfs_s3_config: '{"identities":[{"name":"anvAdmin","credentials":[{"accessKey":"snu8yoP6QAlY0ne4","secretKey":"PNzBcmeLNEdR0oviwm04NQAicOrDH1Km"}],"actions":["Admin","Read","Write"]},{"name":"anvReadOnly","credentials":[{"accessKey":"SCigFee6c5lbi04A","secretKey":"kgFhbT38R8WUYVtiFQ1OiSVOrYr3NKku"}],"actions":["Read"]}]}' ``` +## Admin Component + +The admin component provides a modern web-based administration interface for managing SeaweedFS clusters. It includes: + +- **Dashboard**: Real-time cluster status and metrics +- **Volume Management**: Monitor volume servers, capacity, and health +- **File Browser**: Browse and manage files in the filer +- **Maintenance Operations**: Trigger maintenance tasks via workers +- **Object Store Management**: Create and manage buckets with web interface + +### Enabling Admin + +To enable the admin interface, add the following to your values.yaml: + +```yaml +admin: + enabled: true + port: 23646 + grpcPort: 33646 # For worker connections + adminUser: "admin" + adminPassword: "your-secure-password" # Leave empty to disable auth + + # Optional: persist admin data + data: + type: "persistentVolumeClaim" + size: "10Gi" + storageClass: "your-storage-class" + + # Optional: enable ingress + ingress: + enabled: true + host: "admin.seaweedfs.local" + className: "nginx" +``` + +The admin interface will be available at `http://:23646` (or via ingress). Workers connect to the admin server via gRPC on port `33646`. + +### Admin Authentication + +If `adminPassword` is set, the admin interface requires authentication: +- Username: Value of `adminUser` (default: `admin`) +- Password: Value of `adminPassword` + +If `adminPassword` is empty or not set, the admin interface runs without authentication (not recommended for production). + +### Admin Data Persistence + +The admin component can store configuration and maintenance data. You can configure storage in several ways: + +- **emptyDir** (default): Data is lost when pod restarts +- **persistentVolumeClaim**: Data persists across pod restarts +- **hostPath**: Data stored on the host filesystem +- **existingClaim**: Use an existing PVC + +## Worker Component + +Workers are maintenance agents that execute cluster maintenance tasks such as vacuum, volume balancing, and erasure coding. Workers connect to the admin server via gRPC and receive task assignments. + +### Enabling Workers + +To enable workers, add the following to your values.yaml: + +```yaml +worker: + enabled: true + replicas: 2 # Scale based on workload + capabilities: "vacuum,balance,erasure_coding" # Tasks this worker can handle + maxConcurrent: 3 # Maximum concurrent tasks per worker + + # Working directory for task execution + # Default: "/tmp/seaweedfs-worker" + # Note: /tmp is ephemeral - use persistent storage (hostPath/existingClaim) for long-running tasks + workingDir: "/tmp/seaweedfs-worker" + + # Optional: configure admin server address + # If not specified, auto-discovers from admin service in the same namespace by looking for + # a service named "-admin" (e.g., "seaweedfs-admin"). + # Auto-discovery only works if the admin is in the same namespace and same Helm release. + # For cross-namespace or separate release scenarios, explicitly set this value. + # Example: If main SeaweedFS is deployed in "production" namespace: + # adminServer: "seaweedfs-admin.production.svc:33646" + adminServer: "" + + # Workers need storage for task execution + # Note: Workers use a Deployment, which does not support `volumeClaimTemplates` + # for dynamic PVC creation per pod. To use persistent storage, you must + # pre-provision a PersistentVolumeClaim and use `type: "existingClaim"`. + data: + type: "emptyDir" # Options: "emptyDir", "hostPath", or "existingClaim" + hostPathPrefix: /storage # For hostPath + # claimName: "worker-pvc" # For existingClaim with pre-provisioned PVC + + # Resource limits for worker pods + resources: + requests: + cpu: "500m" + memory: "512Mi" + limits: + cpu: "2" + memory: "2Gi" +``` + +### Worker Capabilities + +Workers can be configured with different capabilities: +- **vacuum**: Reclaim deleted file space +- **balance**: Balance volumes across volume servers +- **erasure_coding**: Handle erasure coding operations + +You can configure workers with all capabilities or create specialized worker pools with specific capabilities. + +### Worker Deployment Strategy + +For production deployments, consider: + +1. **Multiple Workers**: Deploy 2+ worker replicas for high availability +2. **Resource Allocation**: Workers need sufficient CPU/memory for maintenance tasks +3. **Storage**: Workers need temporary storage for vacuum and balance operations (size depends on volume size) +4. **Specialized Workers**: Create separate worker deployments for different capabilities if needed + +Example specialized worker configuration: + +For specialized worker pools, deploy separate Helm releases with different capabilities: + +**values-worker-vacuum.yaml** (for vacuum operations): +```yaml +# Disable all other components, enable only workers +master: + enabled: false +volume: + enabled: false +filer: + enabled: false +s3: + enabled: false +admin: + enabled: false + +worker: + enabled: true + replicas: 2 + capabilities: "vacuum" + maxConcurrent: 2 + # REQUIRED: Point to the admin service of your main SeaweedFS release + # Replace with the namespace where your main seaweedfs is deployed + # Example: If deploying in namespace "production": + # adminServer: "seaweedfs-admin.production.svc:33646" + adminServer: "seaweedfs-admin..svc:33646" +``` + +**values-worker-balance.yaml** (for balance operations): +```yaml +# Disable all other components, enable only workers +master: + enabled: false +volume: + enabled: false +filer: + enabled: false +s3: + enabled: false +admin: + enabled: false + +worker: + enabled: true + replicas: 1 + capabilities: "balance" + maxConcurrent: 1 + # REQUIRED: Point to the admin service of your main SeaweedFS release + # Replace with the namespace where your main seaweedfs is deployed + # Example: If deploying in namespace "production": + # adminServer: "seaweedfs-admin.production.svc:33646" + adminServer: "seaweedfs-admin..svc:33646" +``` + +Deploy the specialized workers as separate releases: +```bash +# Deploy vacuum workers +helm install seaweedfs-worker-vacuum seaweedfs/seaweedfs -f values-worker-vacuum.yaml + +# Deploy balance workers +helm install seaweedfs-worker-balance seaweedfs/seaweedfs -f values-worker-balance.yaml +``` + ## Enterprise For enterprise users, please visit [seaweedfs.com](https://seaweedfs.com) for the SeaweedFS Enterprise Edition, diff --git a/k8s/charts/seaweedfs/templates/admin/admin-ingress.yaml b/k8s/charts/seaweedfs/templates/admin/admin-ingress.yaml new file mode 100644 index 000000000..216ef8a86 --- /dev/null +++ b/k8s/charts/seaweedfs/templates/admin/admin-ingress.yaml @@ -0,0 +1,52 @@ +{{- if and .Values.admin.enabled .Values.admin.ingress.enabled }} +{{- if semverCompare ">=1.19-0" .Capabilities.KubeVersion.GitVersion }} +apiVersion: networking.k8s.io/v1 +{{- else if semverCompare ">=1.14-0" .Capabilities.KubeVersion.GitVersion }} +apiVersion: networking.k8s.io/v1beta1 +{{- else }} +apiVersion: extensions/v1beta1 +{{- end }} +kind: Ingress +metadata: + name: ingress-{{ template "seaweedfs.name" . }}-admin + namespace: {{ .Release.Namespace }} + annotations: + {{- if and (not (semverCompare ">=1.18-0" .Capabilities.KubeVersion.GitVersion)) .Values.admin.ingress.className }} + kubernetes.io/ingress.class: {{ .Values.admin.ingress.className }} + {{- end }} + {{- with .Values.admin.ingress.annotations }} + {{- toYaml . | nindent 4 }} + {{- end }} + labels: + app.kubernetes.io/name: {{ template "seaweedfs.name" . }} + helm.sh/chart: {{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }} + app.kubernetes.io/managed-by: {{ .Release.Service }} + app.kubernetes.io/instance: {{ .Release.Name }} + app.kubernetes.io/component: admin +spec: + {{- if and (semverCompare ">=1.18-0" .Capabilities.KubeVersion.GitVersion) .Values.admin.ingress.className }} + ingressClassName: {{ .Values.admin.ingress.className | quote }} + {{- end }} + tls: + {{ .Values.admin.ingress.tls | default list | toYaml | nindent 6}} + rules: + - {{- if .Values.admin.ingress.host }} + host: {{ .Values.admin.ingress.host | quote }} + {{- end }} + http: + paths: + - path: {{ .Values.admin.ingress.path | quote }} + {{- if semverCompare ">=1.18-0" .Capabilities.KubeVersion.GitVersion }} + pathType: {{ .Values.admin.ingress.pathType | quote }} + {{- end }} + backend: +{{- if semverCompare ">=1.19-0" .Capabilities.KubeVersion.GitVersion }} + service: + name: {{ template "seaweedfs.name" . }}-admin + port: + number: {{ .Values.admin.port }} +{{- else }} + serviceName: {{ template "seaweedfs.name" . }}-admin + servicePort: {{ .Values.admin.port }} +{{- end }} +{{- end }} diff --git a/k8s/charts/seaweedfs/templates/admin/admin-service.yaml b/k8s/charts/seaweedfs/templates/admin/admin-service.yaml new file mode 100644 index 000000000..825049a44 --- /dev/null +++ b/k8s/charts/seaweedfs/templates/admin/admin-service.yaml @@ -0,0 +1,39 @@ +{{- if .Values.admin.enabled }} +apiVersion: v1 +kind: Service +metadata: + name: {{ template "seaweedfs.name" . }}-admin + namespace: {{ .Release.Namespace }} + labels: + app.kubernetes.io/name: {{ template "seaweedfs.name" . }} + helm.sh/chart: {{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }} + app.kubernetes.io/managed-by: {{ .Release.Service }} + app.kubernetes.io/instance: {{ .Release.Name }} + app.kubernetes.io/component: admin +{{- if .Values.admin.service.annotations }} + annotations: + {{- toYaml .Values.admin.service.annotations | nindent 4 }} +{{- end }} +spec: + type: {{ .Values.admin.service.type }} + ports: + - name: "http" + port: {{ .Values.admin.port }} + targetPort: {{ .Values.admin.port }} + protocol: TCP + - name: "grpc" + port: {{ .Values.admin.grpcPort }} + targetPort: {{ .Values.admin.grpcPort }} + protocol: TCP + {{- if .Values.admin.metricsPort }} + - name: "metrics" + port: {{ .Values.admin.metricsPort }} + targetPort: {{ .Values.admin.metricsPort }} + protocol: TCP + {{- end }} + selector: + app.kubernetes.io/name: {{ template "seaweedfs.name" . }} + app.kubernetes.io/instance: {{ .Release.Name }} + app.kubernetes.io/component: admin +{{- end }} + diff --git a/k8s/charts/seaweedfs/templates/admin/admin-servicemonitor.yaml b/k8s/charts/seaweedfs/templates/admin/admin-servicemonitor.yaml new file mode 100644 index 000000000..271197a3a --- /dev/null +++ b/k8s/charts/seaweedfs/templates/admin/admin-servicemonitor.yaml @@ -0,0 +1,33 @@ +{{- if .Values.admin.enabled }} +{{- if .Values.admin.metricsPort }} +{{- if .Values.global.monitoring.enabled }} +apiVersion: monitoring.coreos.com/v1 +kind: ServiceMonitor +metadata: + name: {{ template "seaweedfs.name" . }}-admin + namespace: {{ .Release.Namespace }} + labels: + app.kubernetes.io/name: {{ template "seaweedfs.name" . }} + helm.sh/chart: {{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }} + app.kubernetes.io/managed-by: {{ .Release.Service }} + app.kubernetes.io/instance: {{ .Release.Name }} + app.kubernetes.io/component: admin + {{- with .Values.global.monitoring.additionalLabels }} + {{- toYaml . | nindent 4 }} + {{- end }} +{{- with .Values.admin.serviceMonitor.annotations }} + annotations: + {{- toYaml . | nindent 4 }} +{{- end }} +spec: + endpoints: + - interval: 30s + port: metrics + scrapeTimeout: 5s + selector: + matchLabels: + app.kubernetes.io/name: {{ template "seaweedfs.name" . }} + app.kubernetes.io/component: admin +{{- end }} +{{- end }} +{{- end }} diff --git a/k8s/charts/seaweedfs/templates/admin/admin-statefulset.yaml b/k8s/charts/seaweedfs/templates/admin/admin-statefulset.yaml new file mode 100644 index 000000000..5b6757439 --- /dev/null +++ b/k8s/charts/seaweedfs/templates/admin/admin-statefulset.yaml @@ -0,0 +1,319 @@ +{{- if .Values.admin.enabled }} +{{- if and (not .Values.admin.masters) (not .Values.global.masterServer) (not .Values.master.enabled) }} +{{- fail "admin.masters or global.masterServer must be set if master.enabled is false" -}} +{{- end }} +apiVersion: apps/v1 +kind: StatefulSet +metadata: + name: {{ template "seaweedfs.name" . }}-admin + namespace: {{ .Release.Namespace }} + labels: + app.kubernetes.io/name: {{ template "seaweedfs.name" . }} + helm.sh/chart: {{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }} + app.kubernetes.io/managed-by: {{ .Release.Service }} + app.kubernetes.io/instance: {{ .Release.Name }} + app.kubernetes.io/component: admin +{{- if .Values.admin.annotations }} + annotations: + {{- toYaml .Values.admin.annotations | nindent 4 }} +{{- end }} +spec: + serviceName: {{ template "seaweedfs.name" . }}-admin + podManagementPolicy: {{ .Values.admin.podManagementPolicy }} + replicas: {{ .Values.admin.replicas }} + selector: + matchLabels: + app.kubernetes.io/name: {{ template "seaweedfs.name" . }} + app.kubernetes.io/instance: {{ .Release.Name }} + app.kubernetes.io/component: admin + template: + metadata: + labels: + app.kubernetes.io/name: {{ template "seaweedfs.name" . }} + helm.sh/chart: {{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }} + app.kubernetes.io/instance: {{ .Release.Name }} + app.kubernetes.io/component: admin + {{ with .Values.podLabels }} + {{- toYaml . | nindent 8 }} + {{- end }} + {{- with .Values.admin.podLabels }} + {{- toYaml . | nindent 8 }} + {{- end }} + annotations: + {{ with .Values.podAnnotations }} + {{- toYaml . | nindent 8 }} + {{- end }} + {{- with .Values.admin.podAnnotations }} + {{- toYaml . | nindent 8 }} + {{- end }} + spec: + restartPolicy: {{ default .Values.global.restartPolicy .Values.admin.restartPolicy }} + {{- if .Values.admin.affinity }} + affinity: + {{ tpl .Values.admin.affinity . | nindent 8 | trim }} + {{- end }} + {{- if .Values.admin.topologySpreadConstraints }} + topologySpreadConstraints: + {{ tpl .Values.admin.topologySpreadConstraints . | nindent 8 | trim }} + {{- end }} + {{- if .Values.admin.tolerations }} + tolerations: + {{ tpl .Values.admin.tolerations . | nindent 8 | trim }} + {{- end }} + {{- include "seaweedfs.imagePullSecrets" . | nindent 6 }} + terminationGracePeriodSeconds: 30 + {{- if .Values.admin.priorityClassName }} + priorityClassName: {{ .Values.admin.priorityClassName | quote }} + {{- end }} + enableServiceLinks: false + {{- if .Values.admin.serviceAccountName }} + serviceAccountName: {{ .Values.admin.serviceAccountName | quote }} + {{- end }} + {{- if .Values.admin.initContainers }} + initContainers: + {{ tpl .Values.admin.initContainers . | nindent 8 | trim }} + {{- end }} + {{- if .Values.admin.podSecurityContext.enabled }} + securityContext: {{- omit .Values.admin.podSecurityContext "enabled" | toYaml | nindent 8 }} + {{- end }} + containers: + - name: seaweedfs + image: {{ template "admin.image" . }} + imagePullPolicy: {{ default "IfNotPresent" .Values.global.imagePullPolicy }} + env: + - name: POD_IP + valueFrom: + fieldRef: + fieldPath: status.podIP + - name: POD_NAME + valueFrom: + fieldRef: + fieldPath: metadata.name + - name: NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace + - name: SEAWEEDFS_FULLNAME + value: "{{ template "seaweedfs.name" . }}" + {{- if .Values.admin.extraEnvironmentVars }} + {{- range $key, $value := .Values.admin.extraEnvironmentVars }} + - name: {{ $key }} + {{- if kindIs "string" $value }} + value: {{ $value | quote }} + {{- else }} + valueFrom: + {{ toYaml $value | nindent 16 | trim }} + {{- end -}} + {{- end }} + {{- end }} + {{- if .Values.global.extraEnvironmentVars }} + {{- range $key, $value := .Values.global.extraEnvironmentVars }} + - name: {{ $key }} + {{- if kindIs "string" $value }} + value: {{ $value | quote }} + {{- else }} + valueFrom: + {{ toYaml $value | nindent 16 | trim }} + {{- end -}} + {{- end }} + {{- end }} + command: + - "/bin/sh" + - "-ec" + - | + exec /usr/bin/weed \ + {{- if or (eq .Values.admin.logs.type "hostPath") (eq .Values.admin.logs.type "persistentVolumeClaim") (eq .Values.admin.logs.type "emptyDir") (eq .Values.admin.logs.type "existingClaim") }} + -logdir=/logs \ + {{- else }} + -logtostderr=true \ + {{- end }} + {{- if .Values.admin.loggingOverrideLevel }} + -v={{ .Values.admin.loggingOverrideLevel }} \ + {{- else }} + -v={{ .Values.global.loggingLevel }} \ + {{- end }} + admin \ + -port={{ .Values.admin.port }} \ + -port.grpc={{ .Values.admin.grpcPort }} \ + {{- if or (eq .Values.admin.data.type "hostPath") (eq .Values.admin.data.type "persistentVolumeClaim") (eq .Values.admin.data.type "emptyDir") (eq .Values.admin.data.type "existingClaim") }} + -dataDir=/data \ + {{- else if .Values.admin.dataDir }} + -dataDir={{ .Values.admin.dataDir }} \ + {{- end }} + {{- if .Values.admin.adminPassword }} + -adminUser={{ .Values.admin.adminUser }} \ + -adminPassword={{ .Values.admin.adminPassword }} \ + {{- end }} + {{- if .Values.admin.masters }} + -masters={{ .Values.admin.masters }}{{- if .Values.admin.extraArgs }} \{{ end }} + {{- else if .Values.global.masterServer }} + -masters={{ .Values.global.masterServer }}{{- if .Values.admin.extraArgs }} \{{ end }} + {{- else }} + -masters={{ range $index := until (.Values.master.replicas | int) }}${SEAWEEDFS_FULLNAME}-master-{{ $index }}.${SEAWEEDFS_FULLNAME}-master.{{ $.Release.Namespace }}:{{ $.Values.master.port }}{{ if lt $index (sub ($.Values.master.replicas | int) 1) }},{{ end }}{{ end }}{{- if .Values.admin.extraArgs }} \{{ end }} + {{- end }} + {{- range $index, $arg := .Values.admin.extraArgs }} + {{ $arg }}{{- if lt $index (sub (len $.Values.admin.extraArgs) 1) }} \{{ end }} + {{- end }} + volumeMounts: + {{- if or (eq .Values.admin.data.type "hostPath") (eq .Values.admin.data.type "persistentVolumeClaim") (eq .Values.admin.data.type "emptyDir") (eq .Values.admin.data.type "existingClaim") }} + - name: admin-data + mountPath: /data + {{- end }} + {{- if or (eq .Values.admin.logs.type "hostPath") (eq .Values.admin.logs.type "persistentVolumeClaim") (eq .Values.admin.logs.type "emptyDir") (eq .Values.admin.logs.type "existingClaim") }} + - name: admin-logs + mountPath: /logs + {{- end }} + {{- if .Values.global.enableSecurity }} + - name: security-config + readOnly: true + mountPath: /etc/seaweedfs/security.toml + subPath: security.toml + - name: ca-cert + readOnly: true + mountPath: /usr/local/share/ca-certificates/ca/ + - name: master-cert + readOnly: true + mountPath: /usr/local/share/ca-certificates/master/ + - name: volume-cert + readOnly: true + mountPath: /usr/local/share/ca-certificates/volume/ + - name: filer-cert + readOnly: true + mountPath: /usr/local/share/ca-certificates/filer/ + - name: client-cert + readOnly: true + mountPath: /usr/local/share/ca-certificates/client/ + {{- end }} + {{ tpl .Values.admin.extraVolumeMounts . | nindent 12 | trim }} + ports: + - containerPort: {{ .Values.admin.port }} + name: http + - containerPort: {{ .Values.admin.grpcPort }} + name: grpc + {{- if .Values.admin.metricsPort }} + - containerPort: {{ .Values.admin.metricsPort }} + name: metrics + {{- end }} + {{- if .Values.admin.readinessProbe.enabled }} + readinessProbe: + httpGet: + path: {{ .Values.admin.readinessProbe.httpGet.path }} + port: {{ .Values.admin.port }} + scheme: {{ .Values.admin.readinessProbe.httpGet.scheme }} + initialDelaySeconds: {{ .Values.admin.readinessProbe.initialDelaySeconds }} + periodSeconds: {{ .Values.admin.readinessProbe.periodSeconds }} + successThreshold: {{ .Values.admin.readinessProbe.successThreshold }} + failureThreshold: {{ .Values.admin.readinessProbe.failureThreshold }} + timeoutSeconds: {{ .Values.admin.readinessProbe.timeoutSeconds }} + {{- end }} + {{- if .Values.admin.livenessProbe.enabled }} + livenessProbe: + httpGet: + path: {{ .Values.admin.livenessProbe.httpGet.path }} + port: {{ .Values.admin.port }} + scheme: {{ .Values.admin.livenessProbe.httpGet.scheme }} + initialDelaySeconds: {{ .Values.admin.livenessProbe.initialDelaySeconds }} + periodSeconds: {{ .Values.admin.livenessProbe.periodSeconds }} + successThreshold: {{ .Values.admin.livenessProbe.successThreshold }} + failureThreshold: {{ .Values.admin.livenessProbe.failureThreshold }} + timeoutSeconds: {{ .Values.admin.livenessProbe.timeoutSeconds }} + {{- end }} + {{- with .Values.admin.resources }} + resources: + {{- toYaml . | nindent 12 }} + {{- end }} + {{- if .Values.admin.containerSecurityContext.enabled }} + securityContext: {{- omit .Values.admin.containerSecurityContext "enabled" | toYaml | nindent 12 }} + {{- end }} + {{- if .Values.admin.sidecars }} + {{- include "common.tplvalues.render" (dict "value" .Values.admin.sidecars "context" $) | nindent 8 }} + {{- end }} + volumes: + {{- if eq .Values.admin.data.type "hostPath" }} + - name: admin-data + hostPath: + path: {{ .Values.admin.data.hostPathPrefix }}/seaweedfs-admin-data + type: DirectoryOrCreate + {{- end }} + {{- if eq .Values.admin.data.type "emptyDir" }} + - name: admin-data + emptyDir: {} + {{- end }} + {{- if eq .Values.admin.data.type "existingClaim" }} + - name: admin-data + persistentVolumeClaim: + claimName: {{ .Values.admin.data.claimName }} + {{- end }} + {{- if eq .Values.admin.logs.type "hostPath" }} + - name: admin-logs + hostPath: + path: {{ .Values.admin.logs.hostPathPrefix }}/logs/seaweedfs/admin + type: DirectoryOrCreate + {{- end }} + {{- if eq .Values.admin.logs.type "emptyDir" }} + - name: admin-logs + emptyDir: {} + {{- end }} + {{- if eq .Values.admin.logs.type "existingClaim" }} + - name: admin-logs + persistentVolumeClaim: + claimName: {{ .Values.admin.logs.claimName }} + {{- end }} + {{- if .Values.global.enableSecurity }} + - name: security-config + configMap: + name: {{ template "seaweedfs.name" . }}-security-config + - name: ca-cert + secret: + secretName: {{ template "seaweedfs.name" . }}-ca-cert + - name: master-cert + secret: + secretName: {{ template "seaweedfs.name" . }}-master-cert + - name: volume-cert + secret: + secretName: {{ template "seaweedfs.name" . }}-volume-cert + - name: filer-cert + secret: + secretName: {{ template "seaweedfs.name" . }}-filer-cert + - name: client-cert + secret: + secretName: {{ template "seaweedfs.name" . }}-client-cert + {{- end }} + {{ tpl .Values.admin.extraVolumes . | indent 8 | trim }} + {{- if .Values.admin.nodeSelector }} + nodeSelector: + {{ tpl .Values.admin.nodeSelector . | indent 8 | trim }} + {{- end }} + {{- $pvc_exists := include "admin.pvc_exists" . -}} + {{- if $pvc_exists }} + volumeClaimTemplates: + {{- if eq .Values.admin.data.type "persistentVolumeClaim" }} + - metadata: + name: admin-data + {{- with .Values.admin.data.annotations }} + annotations: + {{- toYaml . | nindent 10 }} + {{- end }} + spec: + accessModes: [ "ReadWriteOnce" ] + storageClassName: {{ .Values.admin.data.storageClass }} + resources: + requests: + storage: {{ .Values.admin.data.size }} + {{- end }} + {{- if eq .Values.admin.logs.type "persistentVolumeClaim" }} + - metadata: + name: admin-logs + {{- with .Values.admin.logs.annotations }} + annotations: + {{- toYaml . | nindent 10 }} + {{- end }} + spec: + accessModes: [ "ReadWriteOnce" ] + storageClassName: {{ .Values.admin.logs.storageClass }} + resources: + requests: + storage: {{ .Values.admin.logs.size }} + {{- end }} + {{- end }} +{{- end }} diff --git a/k8s/charts/seaweedfs/templates/shared/_helpers.tpl b/k8s/charts/seaweedfs/templates/shared/_helpers.tpl index d22d14224..28e0b0906 100644 --- a/k8s/charts/seaweedfs/templates/shared/_helpers.tpl +++ b/k8s/charts/seaweedfs/templates/shared/_helpers.tpl @@ -83,6 +83,26 @@ Inject extra environment vars in the format key:value, if populated {{- end -}} {{- end -}} +{{/* Return the proper admin image */}} +{{- define "admin.image" -}} +{{- if .Values.admin.imageOverride -}} +{{- $imageOverride := .Values.admin.imageOverride -}} +{{- printf "%s" $imageOverride -}} +{{- else -}} +{{- include "common.image" . }} +{{- end -}} +{{- end -}} + +{{/* Return the proper worker image */}} +{{- define "worker.image" -}} +{{- if .Values.worker.imageOverride -}} +{{- $imageOverride := .Values.worker.imageOverride -}} +{{- printf "%s" $imageOverride -}} +{{- else -}} +{{- include "common.image" . }} +{{- end -}} +{{- end -}} + {{/* Return the proper volume image */}} {{- define "volume.image" -}} {{- if .Values.volume.imageOverride -}} @@ -136,6 +156,15 @@ Inject extra environment vars in the format key:value, if populated {{- end -}} {{- end -}} +{{/* check if any Admin PVC exists */}} +{{- define "admin.pvc_exists" -}} +{{- if or (eq .Values.admin.data.type "persistentVolumeClaim") (eq .Values.admin.logs.type "persistentVolumeClaim") -}} +{{- printf "true" -}} +{{- else -}} +{{- printf "" -}} +{{- end -}} +{{- end -}} + {{/* check if any InitContainers exist for Volumes */}} {{- define "volume.initContainers_exists" -}} {{- if or (not (empty .Values.volume.idx )) (not (empty .Values.volume.initContainers )) -}} diff --git a/k8s/charts/seaweedfs/templates/worker/worker-deployment.yaml b/k8s/charts/seaweedfs/templates/worker/worker-deployment.yaml new file mode 100644 index 000000000..40c1e32af --- /dev/null +++ b/k8s/charts/seaweedfs/templates/worker/worker-deployment.yaml @@ -0,0 +1,271 @@ +{{- if .Values.worker.enabled }} +{{- if and (not .Values.worker.adminServer) (not .Values.admin.enabled) }} +{{- fail "worker.adminServer must be set if admin.enabled is false within the same release" -}} +{{- end }} +apiVersion: apps/v1 +kind: Deployment +metadata: + name: {{ template "seaweedfs.name" . }}-worker + namespace: {{ .Release.Namespace }} + labels: + app.kubernetes.io/name: {{ template "seaweedfs.name" . }} + helm.sh/chart: {{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }} + app.kubernetes.io/managed-by: {{ .Release.Service }} + app.kubernetes.io/instance: {{ .Release.Name }} + app.kubernetes.io/component: worker +{{- if .Values.worker.annotations }} + annotations: + {{- toYaml .Values.worker.annotations | nindent 4 }} +{{- end }} +spec: + replicas: {{ .Values.worker.replicas }} + selector: + matchLabels: + app.kubernetes.io/name: {{ template "seaweedfs.name" . }} + app.kubernetes.io/instance: {{ .Release.Name }} + app.kubernetes.io/component: worker + template: + metadata: + labels: + app.kubernetes.io/name: {{ template "seaweedfs.name" . }} + helm.sh/chart: {{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }} + app.kubernetes.io/instance: {{ .Release.Name }} + app.kubernetes.io/component: worker + {{ with .Values.podLabels }} + {{- toYaml . | nindent 8 }} + {{- end }} + {{- with .Values.worker.podLabels }} + {{- toYaml . | nindent 8 }} + {{- end }} + annotations: + {{ with .Values.podAnnotations }} + {{- toYaml . | nindent 8 }} + {{- end }} + {{- with .Values.worker.podAnnotations }} + {{- toYaml . | nindent 8 }} + {{- end }} + spec: + restartPolicy: {{ default .Values.global.restartPolicy .Values.worker.restartPolicy }} + {{- if .Values.worker.affinity }} + affinity: + {{ tpl .Values.worker.affinity . | nindent 8 | trim }} + {{- end }} + {{- if .Values.worker.topologySpreadConstraints }} + topologySpreadConstraints: + {{ tpl .Values.worker.topologySpreadConstraints . | nindent 8 | trim }} + {{- end }} + {{- if .Values.worker.tolerations }} + tolerations: + {{ tpl .Values.worker.tolerations . | nindent 8 | trim }} + {{- end }} + {{- include "seaweedfs.imagePullSecrets" . | nindent 6 }} + terminationGracePeriodSeconds: 60 + {{- if .Values.worker.priorityClassName }} + priorityClassName: {{ .Values.worker.priorityClassName | quote }} + {{- end }} + enableServiceLinks: false + {{- if .Values.worker.serviceAccountName }} + serviceAccountName: {{ .Values.worker.serviceAccountName | quote }} + {{- end }} + {{- if .Values.worker.initContainers }} + initContainers: + {{ tpl .Values.worker.initContainers . | nindent 8 | trim }} + {{- end }} + {{- if .Values.worker.podSecurityContext.enabled }} + securityContext: {{- omit .Values.worker.podSecurityContext "enabled" | toYaml | nindent 8 }} + {{- end }} + containers: + - name: seaweedfs + image: {{ template "worker.image" . }} + imagePullPolicy: {{ default "IfNotPresent" .Values.global.imagePullPolicy }} + env: + - name: POD_IP + valueFrom: + fieldRef: + fieldPath: status.podIP + - name: POD_NAME + valueFrom: + fieldRef: + fieldPath: metadata.name + - name: NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace + - name: SEAWEEDFS_FULLNAME + value: "{{ template "seaweedfs.name" . }}" + {{- if .Values.worker.extraEnvironmentVars }} + {{- range $key, $value := .Values.worker.extraEnvironmentVars }} + - name: {{ $key }} + {{- if kindIs "string" $value }} + value: {{ $value | quote }} + {{- else }} + valueFrom: + {{ toYaml $value | nindent 16 | trim }} + {{- end -}} + {{- end }} + {{- end }} + {{- if .Values.global.extraEnvironmentVars }} + {{- range $key, $value := .Values.global.extraEnvironmentVars }} + - name: {{ $key }} + {{- if kindIs "string" $value }} + value: {{ $value | quote }} + {{- else }} + valueFrom: + {{ toYaml $value | nindent 16 | trim }} + {{- end -}} + {{- end }} + {{- end }} + command: + - "/bin/sh" + - "-ec" + - | + exec /usr/bin/weed \ + {{- if or (eq .Values.worker.logs.type "hostPath") (eq .Values.worker.logs.type "emptyDir") (eq .Values.worker.logs.type "existingClaim") }} + -logdir=/logs \ + {{- else }} + -logtostderr=true \ + {{- end }} + {{- if .Values.worker.loggingOverrideLevel }} + -v={{ .Values.worker.loggingOverrideLevel }} \ + {{- else }} + -v={{ .Values.global.loggingLevel }} \ + {{- end }} + worker \ + {{- if .Values.worker.adminServer }} + -adminServer={{ .Values.worker.adminServer }} \ + {{- else }} + -adminServer={{ template "seaweedfs.name" . }}-admin.{{ .Release.Namespace }}:{{ .Values.admin.grpcPort }} \ + {{- end }} + -capabilities={{ .Values.worker.capabilities }} \ + -maxConcurrent={{ .Values.worker.maxConcurrent }} \ + -workingDir={{ .Values.worker.workingDir }}{{- if .Values.worker.extraArgs }} \{{ end }} + {{- range $index, $arg := .Values.worker.extraArgs }} + {{ $arg }}{{- if lt $index (sub (len $.Values.worker.extraArgs) 1) }} \{{ end }} + {{- end }} + volumeMounts: + {{- if or (eq .Values.worker.data.type "hostPath") (eq .Values.worker.data.type "emptyDir") (eq .Values.worker.data.type "existingClaim") }} + - name: worker-data + mountPath: {{ .Values.worker.workingDir }} + {{- end }} + {{- if or (eq .Values.worker.logs.type "hostPath") (eq .Values.worker.logs.type "emptyDir") (eq .Values.worker.logs.type "existingClaim") }} + - name: worker-logs + mountPath: /logs + {{- end }} + {{- if .Values.global.enableSecurity }} + - name: security-config + readOnly: true + mountPath: /etc/seaweedfs/security.toml + subPath: security.toml + - name: ca-cert + readOnly: true + mountPath: /usr/local/share/ca-certificates/ca/ + - name: master-cert + readOnly: true + mountPath: /usr/local/share/ca-certificates/master/ + - name: volume-cert + readOnly: true + mountPath: /usr/local/share/ca-certificates/volume/ + - name: filer-cert + readOnly: true + mountPath: /usr/local/share/ca-certificates/filer/ + - name: client-cert + readOnly: true + mountPath: /usr/local/share/ca-certificates/client/ + {{- end }} + {{ tpl .Values.worker.extraVolumeMounts . | nindent 12 | trim }} + ports: + {{- if .Values.worker.metricsPort }} + - containerPort: {{ .Values.worker.metricsPort }} + name: metrics + {{- end }} + {{- with .Values.worker.resources }} + resources: + {{- toYaml . | nindent 12 }} + {{- end }} + {{- if .Values.worker.livenessProbe.enabled }} + livenessProbe: + {{- with .Values.worker.livenessProbe.tcpSocket }} + tcpSocket: + port: {{ .port }} + {{- end }} + initialDelaySeconds: {{ .Values.worker.livenessProbe.initialDelaySeconds }} + periodSeconds: {{ .Values.worker.livenessProbe.periodSeconds }} + successThreshold: {{ .Values.worker.livenessProbe.successThreshold }} + failureThreshold: {{ .Values.worker.livenessProbe.failureThreshold }} + timeoutSeconds: {{ .Values.worker.livenessProbe.timeoutSeconds }} + {{- end }} + {{- if .Values.worker.readinessProbe.enabled }} + readinessProbe: + {{- with .Values.worker.readinessProbe.tcpSocket }} + tcpSocket: + port: {{ .port }} + {{- end }} + initialDelaySeconds: {{ .Values.worker.readinessProbe.initialDelaySeconds }} + periodSeconds: {{ .Values.worker.readinessProbe.periodSeconds }} + successThreshold: {{ .Values.worker.readinessProbe.successThreshold }} + failureThreshold: {{ .Values.worker.readinessProbe.failureThreshold }} + timeoutSeconds: {{ .Values.worker.readinessProbe.timeoutSeconds }} + {{- end }} + {{- if .Values.worker.containerSecurityContext.enabled }} + securityContext: {{- omit .Values.worker.containerSecurityContext "enabled" | toYaml | nindent 12 }} + {{- end }} + {{- if .Values.worker.sidecars }} + {{- include "common.tplvalues.render" (dict "value" .Values.worker.sidecars "context" $) | nindent 8 }} + {{- end }} + volumes: + {{- if eq .Values.worker.data.type "hostPath" }} + - name: worker-data + hostPath: + path: {{ .Values.worker.data.hostPathPrefix }}/seaweedfs-worker-data + type: DirectoryOrCreate + {{- end }} + {{- if eq .Values.worker.data.type "emptyDir" }} + - name: worker-data + emptyDir: {} + {{- end }} + {{- if eq .Values.worker.data.type "existingClaim" }} + - name: worker-data + persistentVolumeClaim: + claimName: {{ .Values.worker.data.claimName }} + {{- end }} + {{- if eq .Values.worker.logs.type "hostPath" }} + - name: worker-logs + hostPath: + path: {{ .Values.worker.logs.hostPathPrefix }}/logs/seaweedfs/worker + type: DirectoryOrCreate + {{- end }} + {{- if eq .Values.worker.logs.type "emptyDir" }} + - name: worker-logs + emptyDir: {} + {{- end }} + {{- if eq .Values.worker.logs.type "existingClaim" }} + - name: worker-logs + persistentVolumeClaim: + claimName: {{ .Values.worker.logs.claimName }} + {{- end }} + {{- if .Values.global.enableSecurity }} + - name: security-config + configMap: + name: {{ template "seaweedfs.name" . }}-security-config + - name: ca-cert + secret: + secretName: {{ template "seaweedfs.name" . }}-ca-cert + - name: master-cert + secret: + secretName: {{ template "seaweedfs.name" . }}-master-cert + - name: volume-cert + secret: + secretName: {{ template "seaweedfs.name" . }}-volume-cert + - name: filer-cert + secret: + secretName: {{ template "seaweedfs.name" . }}-filer-cert + - name: client-cert + secret: + secretName: {{ template "seaweedfs.name" . }}-client-cert + {{- end }} + {{ tpl .Values.worker.extraVolumes . | indent 8 | trim }} + {{- if .Values.worker.nodeSelector }} + nodeSelector: + {{ tpl .Values.worker.nodeSelector . | indent 8 | trim }} + {{- end }} +{{- end }} diff --git a/k8s/charts/seaweedfs/templates/worker/worker-service.yaml b/k8s/charts/seaweedfs/templates/worker/worker-service.yaml new file mode 100644 index 000000000..cf9885e2f --- /dev/null +++ b/k8s/charts/seaweedfs/templates/worker/worker-service.yaml @@ -0,0 +1,26 @@ +{{- if .Values.worker.enabled }} +apiVersion: v1 +kind: Service +metadata: + name: {{ template "seaweedfs.name" . }}-worker + namespace: {{ .Release.Namespace }} + labels: + app.kubernetes.io/name: {{ template "seaweedfs.name" . }} + helm.sh/chart: {{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }} + app.kubernetes.io/managed-by: {{ .Release.Service }} + app.kubernetes.io/instance: {{ .Release.Name }} + app.kubernetes.io/component: worker +spec: + clusterIP: None # Headless service + {{- if .Values.worker.metricsPort }} + ports: + - name: "metrics" + port: {{ .Values.worker.metricsPort }} + targetPort: {{ .Values.worker.metricsPort }} + protocol: TCP + {{- end }} + selector: + app.kubernetes.io/name: {{ template "seaweedfs.name" . }} + app.kubernetes.io/instance: {{ .Release.Name }} + app.kubernetes.io/component: worker +{{- end }} diff --git a/k8s/charts/seaweedfs/templates/worker/worker-servicemonitor.yaml b/k8s/charts/seaweedfs/templates/worker/worker-servicemonitor.yaml new file mode 100644 index 000000000..7f9590dab --- /dev/null +++ b/k8s/charts/seaweedfs/templates/worker/worker-servicemonitor.yaml @@ -0,0 +1,33 @@ +{{- if .Values.worker.enabled }} +{{- if .Values.worker.metricsPort }} +{{- if .Values.global.monitoring.enabled }} +apiVersion: monitoring.coreos.com/v1 +kind: ServiceMonitor +metadata: + name: {{ template "seaweedfs.name" . }}-worker + namespace: {{ .Release.Namespace }} + labels: + app.kubernetes.io/name: {{ template "seaweedfs.name" . }} + helm.sh/chart: {{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }} + app.kubernetes.io/managed-by: {{ .Release.Service }} + app.kubernetes.io/instance: {{ .Release.Name }} + app.kubernetes.io/component: worker + {{- with .Values.global.monitoring.additionalLabels }} + {{- toYaml . | nindent 4 }} + {{- end }} +{{- with .Values.worker.serviceMonitor.annotations }} + annotations: + {{- toYaml . | nindent 4 }} +{{- end }} +spec: + endpoints: + - interval: 30s + port: metrics + scrapeTimeout: 5s + selector: + matchLabels: + app.kubernetes.io/name: {{ template "seaweedfs.name" . }} + app.kubernetes.io/component: worker +{{- end }} +{{- end }} +{{- end }} diff --git a/k8s/charts/seaweedfs/values.yaml b/k8s/charts/seaweedfs/values.yaml index 547b05479..a250c56ec 100644 --- a/k8s/charts/seaweedfs/values.yaml +++ b/k8s/charts/seaweedfs/values.yaml @@ -1092,6 +1092,238 @@ sftp: failureThreshold: 100 timeoutSeconds: 10 +admin: + enabled: false + imageOverride: null + restartPolicy: null + replicas: 1 + port: 23646 # Default admin port + grpcPort: 33646 # Default gRPC port for worker connections + metricsPort: 9327 + loggingOverrideLevel: null + + # Admin authentication + adminUser: "admin" + adminPassword: "" # If empty, auth is disabled + + # Data directory for admin configuration and maintenance data + dataDir: "" # If empty, configuration is kept in memory only + + # Master servers to connect to + # If empty, uses global.masterServer or auto-discovers from master statefulset + masters: "" + + # Custom command line arguments to add to the admin command + # Example: ["-customFlag", "value", "-anotherFlag"] + extraArgs: [] + + # Storage configuration + data: + type: "emptyDir" # Options: "hostPath", "persistentVolumeClaim", "emptyDir", "existingClaim" + size: "10Gi" + storageClass: "" + hostPathPrefix: /storage + claimName: "" + annotations: {} + + logs: + type: "emptyDir" # Options: "hostPath", "persistentVolumeClaim", "emptyDir", "existingClaim" + size: "5Gi" + storageClass: "" + hostPathPrefix: /storage + claimName: "" + annotations: {} + + # Additional resources + sidecars: [] + initContainers: "" + extraVolumes: "" + extraVolumeMounts: "" + podLabels: {} + podAnnotations: {} + annotations: {} + + ## Set podManagementPolicy + podManagementPolicy: Parallel + + # Affinity Settings + # Commenting out or setting as empty the affinity variable, will allow + # deployment to single node services such as Minikube + affinity: | + podAntiAffinity: + requiredDuringSchedulingIgnoredDuringExecution: + - labelSelector: + matchLabels: + app.kubernetes.io/name: {{ template "seaweedfs.name" . }} + app.kubernetes.io/instance: {{ .Release.Name }} + app.kubernetes.io/component: admin + topologyKey: kubernetes.io/hostname + + # Topology Spread Constraints Settings + # This should map directly to the value of the topologySpreadConstraints + # for a PodSpec. By Default no constraints are set. + topologySpreadConstraints: "" + + resources: {} + tolerations: "" + nodeSelector: "" + priorityClassName: "" + serviceAccountName: "" + podSecurityContext: {} + containerSecurityContext: {} + + extraEnvironmentVars: {} + + # Health checks + livenessProbe: + enabled: true + httpGet: + path: /health + scheme: HTTP + initialDelaySeconds: 20 + periodSeconds: 60 + successThreshold: 1 + failureThreshold: 5 + timeoutSeconds: 10 + + readinessProbe: + enabled: true + httpGet: + path: /health + scheme: HTTP + initialDelaySeconds: 15 + periodSeconds: 15 + successThreshold: 1 + failureThreshold: 3 + timeoutSeconds: 10 + + ingress: + enabled: false + className: "nginx" + # host: false for "*" hostname + host: "admin.seaweedfs.local" + path: "/" + pathType: Prefix + annotations: {} + tls: [] + + service: + type: ClusterIP + annotations: {} + + # ServiceMonitor annotations (separate from pod/deployment annotations) + serviceMonitor: + annotations: {} + +worker: + enabled: false + imageOverride: null + restartPolicy: null + replicas: 1 + loggingOverrideLevel: null + metricsPort: 9327 + + # Admin server to connect to + # Format: "host:port" or auto-discover from admin service + adminServer: "" + + # Worker capabilities - comma-separated list + # Available: vacuum, balance, erasure_coding + # Default: "vacuum,balance,erasure_coding" (all capabilities) + capabilities: "vacuum,balance,erasure_coding" + + # Maximum number of concurrent tasks + maxConcurrent: 3 + + # Working directory for task execution + workingDir: "/tmp/seaweedfs-worker" + + # Custom command line arguments to add to the worker command + # Example: ["-customFlag", "value", "-anotherFlag"] + extraArgs: [] + + # Storage configuration for working directory + # Note: Workers use Deployment, so use "emptyDir", "hostPath", or "existingClaim" + # Do NOT use "persistentVolumeClaim" - use "existingClaim" with pre-provisioned PVC instead + data: + type: "emptyDir" # Options: "hostPath", "emptyDir", "existingClaim" + hostPathPrefix: /storage + claimName: "" # For existingClaim type + + logs: + type: "emptyDir" # Options: "hostPath", "emptyDir", "existingClaim" + hostPathPrefix: /storage + claimName: "" # For existingClaim type + + # Additional resources + sidecars: [] + initContainers: "" + extraVolumes: "" + extraVolumeMounts: "" + podLabels: {} + podAnnotations: {} + annotations: {} + + # Affinity Settings + # Commenting out or setting as empty the affinity variable, will allow + # deployment to single node services such as Minikube + affinity: | + podAntiAffinity: + requiredDuringSchedulingIgnoredDuringExecution: + - labelSelector: + matchLabels: + app.kubernetes.io/name: {{ template "seaweedfs.name" . }} + app.kubernetes.io/instance: {{ .Release.Name }} + app.kubernetes.io/component: worker + topologyKey: kubernetes.io/hostname + + # Topology Spread Constraints Settings + # This should map directly to the value of the topologySpreadConstraints + # for a PodSpec. By Default no constraints are set. + topologySpreadConstraints: "" + + resources: + requests: + cpu: "500m" + memory: "512Mi" + limits: + cpu: "2" + memory: "2Gi" + tolerations: "" + nodeSelector: "" + priorityClassName: "" + serviceAccountName: "" + podSecurityContext: {} + containerSecurityContext: {} + + extraEnvironmentVars: {} + + # Health checks for worker pods + # Since workers do not have an HTTP endpoint, a tcpSocket probe on the metrics port is recommended. + livenessProbe: + enabled: true + tcpSocket: + port: metrics + initialDelaySeconds: 30 + periodSeconds: 60 + successThreshold: 1 + failureThreshold: 5 + timeoutSeconds: 10 + + readinessProbe: + enabled: true + tcpSocket: + port: metrics + initialDelaySeconds: 20 + periodSeconds: 15 + successThreshold: 1 + failureThreshold: 3 + timeoutSeconds: 10 + + # ServiceMonitor annotations (separate from pod/deployment annotations) + serviceMonitor: + annotations: {} + # All-in-one deployment configuration allInOne: enabled: false diff --git a/weed/command/worker.go b/weed/command/worker.go index 6e592f73f..1a36899e2 100644 --- a/weed/command/worker.go +++ b/weed/command/worker.go @@ -19,6 +19,9 @@ import ( _ "github.com/seaweedfs/seaweedfs/weed/worker/tasks/balance" _ "github.com/seaweedfs/seaweedfs/weed/worker/tasks/erasure_coding" _ "github.com/seaweedfs/seaweedfs/weed/worker/tasks/vacuum" + // TODO: Implement additional task packages: + // _ "github.com/seaweedfs/seaweedfs/weed/worker/tasks/remote" - for uploading volumes to remote/cloud storage + // _ "github.com/seaweedfs/seaweedfs/weed/worker/tasks/replication" - for fixing replication issues and maintaining data consistency ) var cmdWorker = &Command{