You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

224 lines
6.4 KiB

FEATURE: add JWT to HTTP endpoints of Filer and use them in S3 Client - one JWT for reading and one for writing, analogous to how the JWT between Master and Volume Server works - I did not implement IP `whiteList` parameter on the filer Additionally, because http_util.DownloadFile now sets the JWT, the `download` command should now work when `jwt.signing.read` is configured. By looking at the code, I think this case did not work before. ## Docs to be adjusted after a release Page `Amazon-S3-API`: ``` # Authentication with Filer You can use mTLS for the gRPC connection between S3-API-Proxy and the filer, as explained in [Security-Configuration](Security-Configuration) - controlled by the `grpc.*` configuration in `security.toml`. Starting with version XX, it is also possible to authenticate the HTTP operations between the S3-API-Proxy and the Filer (especially uploading new files). This is configured by setting `filer_jwt.signing.key` and `filer_jwt.signing.read.key` in `security.toml`. With both configurations (gRPC and JWT), it is possible to have Filer and S3 communicate in fully authenticated fashion; so Filer will reject any unauthenticated communication. ``` Page `Security Overview`: ``` The following items are not covered, yet: - master server http REST services Starting with version XX, the Filer HTTP REST services can be secured with a JWT, by setting `filer_jwt.signing.key` and `filer_jwt.signing.read.key` in `security.toml`. ... Before version XX: "weed filer -disableHttp", disable http operations, only gRPC operations are allowed. This works with "weed mount" by FUSE. It does **not work** with the [S3 Gateway](Amazon S3 API), as this does HTTP calls to the Filer. Starting with version XX: secured by JWT, by setting `filer_jwt.signing.key` and `filer_jwt.signing.read.key` in `security.toml`. **This now works with the [S3 Gateway](Amazon S3 API).** ... # Securing Filer HTTP with JWT To enable JWT-based access control for the Filer, 1. generate `security.toml` file by `weed scaffold -config=security` 2. set `filer_jwt.signing.key` to a secret string - and optionally filer_jwt.signing.read.key` as well to a secret string 3. copy the same `security.toml` file to the filers and all S3 proxies. If `filer_jwt.signing.key` is configured: When sending upload/update/delete HTTP operations to a filer server, the request header `Authorization` should be the JWT string (`Authorization: Bearer [JwtToken]`). The operation is authorized after the filer validates the JWT with `filer_jwt.signing.key`. If `filer_jwt.signing.read.key` is configured: When sending GET or HEAD requests to a filer server, the request header `Authorization` should be the JWT string (`Authorization: Bearer [JwtToken]`). The operation is authorized after the filer validates the JWT with `filer_jwt.signing.read.key`. The S3 API Gateway reads the above JWT keys and sends authenticated HTTP requests to the filer. ``` Page `Security Configuration`: ``` (update scaffold file) ... [filer_jwt.signing] key = "blahblahblahblah" [filer_jwt.signing.read] key = "blahblahblahblah" ``` Resolves: #158
3 years ago
6 years ago
FEATURE: add JWT to HTTP endpoints of Filer and use them in S3 Client - one JWT for reading and one for writing, analogous to how the JWT between Master and Volume Server works - I did not implement IP `whiteList` parameter on the filer Additionally, because http_util.DownloadFile now sets the JWT, the `download` command should now work when `jwt.signing.read` is configured. By looking at the code, I think this case did not work before. ## Docs to be adjusted after a release Page `Amazon-S3-API`: ``` # Authentication with Filer You can use mTLS for the gRPC connection between S3-API-Proxy and the filer, as explained in [Security-Configuration](Security-Configuration) - controlled by the `grpc.*` configuration in `security.toml`. Starting with version XX, it is also possible to authenticate the HTTP operations between the S3-API-Proxy and the Filer (especially uploading new files). This is configured by setting `filer_jwt.signing.key` and `filer_jwt.signing.read.key` in `security.toml`. With both configurations (gRPC and JWT), it is possible to have Filer and S3 communicate in fully authenticated fashion; so Filer will reject any unauthenticated communication. ``` Page `Security Overview`: ``` The following items are not covered, yet: - master server http REST services Starting with version XX, the Filer HTTP REST services can be secured with a JWT, by setting `filer_jwt.signing.key` and `filer_jwt.signing.read.key` in `security.toml`. ... Before version XX: "weed filer -disableHttp", disable http operations, only gRPC operations are allowed. This works with "weed mount" by FUSE. It does **not work** with the [S3 Gateway](Amazon S3 API), as this does HTTP calls to the Filer. Starting with version XX: secured by JWT, by setting `filer_jwt.signing.key` and `filer_jwt.signing.read.key` in `security.toml`. **This now works with the [S3 Gateway](Amazon S3 API).** ... # Securing Filer HTTP with JWT To enable JWT-based access control for the Filer, 1. generate `security.toml` file by `weed scaffold -config=security` 2. set `filer_jwt.signing.key` to a secret string - and optionally filer_jwt.signing.read.key` as well to a secret string 3. copy the same `security.toml` file to the filers and all S3 proxies. If `filer_jwt.signing.key` is configured: When sending upload/update/delete HTTP operations to a filer server, the request header `Authorization` should be the JWT string (`Authorization: Bearer [JwtToken]`). The operation is authorized after the filer validates the JWT with `filer_jwt.signing.key`. If `filer_jwt.signing.read.key` is configured: When sending GET or HEAD requests to a filer server, the request header `Authorization` should be the JWT string (`Authorization: Bearer [JwtToken]`). The operation is authorized after the filer validates the JWT with `filer_jwt.signing.read.key`. The S3 API Gateway reads the above JWT keys and sends authenticated HTTP requests to the filer. ``` Page `Security Configuration`: ``` (update scaffold file) ... [filer_jwt.signing] key = "blahblahblahblah" [filer_jwt.signing.read] key = "blahblahblahblah" ``` Resolves: #158
3 years ago
4 years ago
4 years ago
4 years ago
4 years ago
4 years ago
FEATURE: add JWT to HTTP endpoints of Filer and use them in S3 Client - one JWT for reading and one for writing, analogous to how the JWT between Master and Volume Server works - I did not implement IP `whiteList` parameter on the filer Additionally, because http_util.DownloadFile now sets the JWT, the `download` command should now work when `jwt.signing.read` is configured. By looking at the code, I think this case did not work before. ## Docs to be adjusted after a release Page `Amazon-S3-API`: ``` # Authentication with Filer You can use mTLS for the gRPC connection between S3-API-Proxy and the filer, as explained in [Security-Configuration](Security-Configuration) - controlled by the `grpc.*` configuration in `security.toml`. Starting with version XX, it is also possible to authenticate the HTTP operations between the S3-API-Proxy and the Filer (especially uploading new files). This is configured by setting `filer_jwt.signing.key` and `filer_jwt.signing.read.key` in `security.toml`. With both configurations (gRPC and JWT), it is possible to have Filer and S3 communicate in fully authenticated fashion; so Filer will reject any unauthenticated communication. ``` Page `Security Overview`: ``` The following items are not covered, yet: - master server http REST services Starting with version XX, the Filer HTTP REST services can be secured with a JWT, by setting `filer_jwt.signing.key` and `filer_jwt.signing.read.key` in `security.toml`. ... Before version XX: "weed filer -disableHttp", disable http operations, only gRPC operations are allowed. This works with "weed mount" by FUSE. It does **not work** with the [S3 Gateway](Amazon S3 API), as this does HTTP calls to the Filer. Starting with version XX: secured by JWT, by setting `filer_jwt.signing.key` and `filer_jwt.signing.read.key` in `security.toml`. **This now works with the [S3 Gateway](Amazon S3 API).** ... # Securing Filer HTTP with JWT To enable JWT-based access control for the Filer, 1. generate `security.toml` file by `weed scaffold -config=security` 2. set `filer_jwt.signing.key` to a secret string - and optionally filer_jwt.signing.read.key` as well to a secret string 3. copy the same `security.toml` file to the filers and all S3 proxies. If `filer_jwt.signing.key` is configured: When sending upload/update/delete HTTP operations to a filer server, the request header `Authorization` should be the JWT string (`Authorization: Bearer [JwtToken]`). The operation is authorized after the filer validates the JWT with `filer_jwt.signing.key`. If `filer_jwt.signing.read.key` is configured: When sending GET or HEAD requests to a filer server, the request header `Authorization` should be the JWT string (`Authorization: Bearer [JwtToken]`). The operation is authorized after the filer validates the JWT with `filer_jwt.signing.read.key`. The S3 API Gateway reads the above JWT keys and sends authenticated HTTP requests to the filer. ``` Page `Security Configuration`: ``` (update scaffold file) ... [filer_jwt.signing] key = "blahblahblahblah" [filer_jwt.signing.read] key = "blahblahblahblah" ``` Resolves: #158
3 years ago
FEATURE: add JWT to HTTP endpoints of Filer and use them in S3 Client - one JWT for reading and one for writing, analogous to how the JWT between Master and Volume Server works - I did not implement IP `whiteList` parameter on the filer Additionally, because http_util.DownloadFile now sets the JWT, the `download` command should now work when `jwt.signing.read` is configured. By looking at the code, I think this case did not work before. ## Docs to be adjusted after a release Page `Amazon-S3-API`: ``` # Authentication with Filer You can use mTLS for the gRPC connection between S3-API-Proxy and the filer, as explained in [Security-Configuration](Security-Configuration) - controlled by the `grpc.*` configuration in `security.toml`. Starting with version XX, it is also possible to authenticate the HTTP operations between the S3-API-Proxy and the Filer (especially uploading new files). This is configured by setting `filer_jwt.signing.key` and `filer_jwt.signing.read.key` in `security.toml`. With both configurations (gRPC and JWT), it is possible to have Filer and S3 communicate in fully authenticated fashion; so Filer will reject any unauthenticated communication. ``` Page `Security Overview`: ``` The following items are not covered, yet: - master server http REST services Starting with version XX, the Filer HTTP REST services can be secured with a JWT, by setting `filer_jwt.signing.key` and `filer_jwt.signing.read.key` in `security.toml`. ... Before version XX: "weed filer -disableHttp", disable http operations, only gRPC operations are allowed. This works with "weed mount" by FUSE. It does **not work** with the [S3 Gateway](Amazon S3 API), as this does HTTP calls to the Filer. Starting with version XX: secured by JWT, by setting `filer_jwt.signing.key` and `filer_jwt.signing.read.key` in `security.toml`. **This now works with the [S3 Gateway](Amazon S3 API).** ... # Securing Filer HTTP with JWT To enable JWT-based access control for the Filer, 1. generate `security.toml` file by `weed scaffold -config=security` 2. set `filer_jwt.signing.key` to a secret string - and optionally filer_jwt.signing.read.key` as well to a secret string 3. copy the same `security.toml` file to the filers and all S3 proxies. If `filer_jwt.signing.key` is configured: When sending upload/update/delete HTTP operations to a filer server, the request header `Authorization` should be the JWT string (`Authorization: Bearer [JwtToken]`). The operation is authorized after the filer validates the JWT with `filer_jwt.signing.key`. If `filer_jwt.signing.read.key` is configured: When sending GET or HEAD requests to a filer server, the request header `Authorization` should be the JWT string (`Authorization: Bearer [JwtToken]`). The operation is authorized after the filer validates the JWT with `filer_jwt.signing.read.key`. The S3 API Gateway reads the above JWT keys and sends authenticated HTTP requests to the filer. ``` Page `Security Configuration`: ``` (update scaffold file) ... [filer_jwt.signing] key = "blahblahblahblah" [filer_jwt.signing.read] key = "blahblahblahblah" ``` Resolves: #158
3 years ago
4 years ago
4 years ago
FEATURE: add JWT to HTTP endpoints of Filer and use them in S3 Client - one JWT for reading and one for writing, analogous to how the JWT between Master and Volume Server works - I did not implement IP `whiteList` parameter on the filer Additionally, because http_util.DownloadFile now sets the JWT, the `download` command should now work when `jwt.signing.read` is configured. By looking at the code, I think this case did not work before. ## Docs to be adjusted after a release Page `Amazon-S3-API`: ``` # Authentication with Filer You can use mTLS for the gRPC connection between S3-API-Proxy and the filer, as explained in [Security-Configuration](Security-Configuration) - controlled by the `grpc.*` configuration in `security.toml`. Starting with version XX, it is also possible to authenticate the HTTP operations between the S3-API-Proxy and the Filer (especially uploading new files). This is configured by setting `filer_jwt.signing.key` and `filer_jwt.signing.read.key` in `security.toml`. With both configurations (gRPC and JWT), it is possible to have Filer and S3 communicate in fully authenticated fashion; so Filer will reject any unauthenticated communication. ``` Page `Security Overview`: ``` The following items are not covered, yet: - master server http REST services Starting with version XX, the Filer HTTP REST services can be secured with a JWT, by setting `filer_jwt.signing.key` and `filer_jwt.signing.read.key` in `security.toml`. ... Before version XX: "weed filer -disableHttp", disable http operations, only gRPC operations are allowed. This works with "weed mount" by FUSE. It does **not work** with the [S3 Gateway](Amazon S3 API), as this does HTTP calls to the Filer. Starting with version XX: secured by JWT, by setting `filer_jwt.signing.key` and `filer_jwt.signing.read.key` in `security.toml`. **This now works with the [S3 Gateway](Amazon S3 API).** ... # Securing Filer HTTP with JWT To enable JWT-based access control for the Filer, 1. generate `security.toml` file by `weed scaffold -config=security` 2. set `filer_jwt.signing.key` to a secret string - and optionally filer_jwt.signing.read.key` as well to a secret string 3. copy the same `security.toml` file to the filers and all S3 proxies. If `filer_jwt.signing.key` is configured: When sending upload/update/delete HTTP operations to a filer server, the request header `Authorization` should be the JWT string (`Authorization: Bearer [JwtToken]`). The operation is authorized after the filer validates the JWT with `filer_jwt.signing.key`. If `filer_jwt.signing.read.key` is configured: When sending GET or HEAD requests to a filer server, the request header `Authorization` should be the JWT string (`Authorization: Bearer [JwtToken]`). The operation is authorized after the filer validates the JWT with `filer_jwt.signing.read.key`. The S3 API Gateway reads the above JWT keys and sends authenticated HTTP requests to the filer. ``` Page `Security Configuration`: ``` (update scaffold file) ... [filer_jwt.signing] key = "blahblahblahblah" [filer_jwt.signing.read] key = "blahblahblahblah" ``` Resolves: #158
3 years ago
  1. package weed_server
  2. import (
  3. "errors"
  4. "net/http"
  5. "os"
  6. "strconv"
  7. "strings"
  8. "sync/atomic"
  9. "time"
  10. "github.com/seaweedfs/seaweedfs/weed/glog"
  11. "github.com/seaweedfs/seaweedfs/weed/security"
  12. "github.com/seaweedfs/seaweedfs/weed/util"
  13. "github.com/seaweedfs/seaweedfs/weed/stats"
  14. )
  15. func (fs *FilerServer) filerHandler(w http.ResponseWriter, r *http.Request) {
  16. start := time.Now()
  17. statusRecorder := stats.NewStatusResponseWriter(w)
  18. w = statusRecorder
  19. origin := r.Header.Get("Origin")
  20. if origin != "" {
  21. if fs.option.AllowedOrigins == nil || len(fs.option.AllowedOrigins) == 0 || fs.option.AllowedOrigins[0] == "*" {
  22. origin = "*"
  23. } else {
  24. originFound := false
  25. for _, allowedOrigin := range fs.option.AllowedOrigins {
  26. if origin == allowedOrigin {
  27. originFound = true
  28. }
  29. }
  30. if !originFound {
  31. writeJsonError(w, r, http.StatusForbidden, errors.New("origin not allowed"))
  32. return
  33. }
  34. }
  35. w.Header().Set("Access-Control-Allow-Origin", origin)
  36. w.Header().Set("Access-Control-Expose-Headers", "*")
  37. w.Header().Set("Access-Control-Allow-Headers", "*")
  38. w.Header().Set("Access-Control-Allow-Credentials", "true")
  39. w.Header().Set("Access-Control-Allow-Methods", "PUT, POST, GET, DELETE, OPTIONS")
  40. }
  41. if r.Method == "OPTIONS" {
  42. OptionsHandler(w, r, false)
  43. return
  44. }
  45. // proxy to volume servers
  46. var fileId string
  47. if strings.HasPrefix(r.RequestURI, "/?proxyChunkId=") {
  48. fileId = r.RequestURI[len("/?proxyChunkId="):]
  49. }
  50. if fileId != "" {
  51. fs.proxyToVolumeServer(w, r, fileId)
  52. stats.FilerHandlerCounter.WithLabelValues(stats.ChunkProxy).Inc()
  53. stats.FilerRequestHistogram.WithLabelValues(stats.ChunkProxy).Observe(time.Since(start).Seconds())
  54. return
  55. }
  56. defer func() {
  57. stats.FilerRequestCounter.WithLabelValues(r.Method, strconv.Itoa(statusRecorder.Status)).Inc()
  58. stats.FilerRequestHistogram.WithLabelValues(r.Method).Observe(time.Since(start).Seconds())
  59. }()
  60. isReadHttpCall := r.Method == "GET" || r.Method == "HEAD"
  61. if !fs.maybeCheckJwtAuthorization(r, !isReadHttpCall) {
  62. writeJsonError(w, r, http.StatusUnauthorized, errors.New("wrong jwt"))
  63. return
  64. }
  65. w.Header().Set("Server", "SeaweedFS Filer "+util.VERSION)
  66. switch r.Method {
  67. case "GET":
  68. fs.GetOrHeadHandler(w, r)
  69. case "HEAD":
  70. fs.GetOrHeadHandler(w, r)
  71. case "DELETE":
  72. if _, ok := r.URL.Query()["tagging"]; ok {
  73. fs.DeleteTaggingHandler(w, r)
  74. } else {
  75. fs.DeleteHandler(w, r)
  76. }
  77. case "POST", "PUT":
  78. // wait until in flight data is less than the limit
  79. contentLength := getContentLength(r)
  80. fs.inFlightDataLimitCond.L.Lock()
  81. inFlightDataSize := atomic.LoadInt64(&fs.inFlightDataSize)
  82. for fs.option.ConcurrentUploadLimit != 0 && inFlightDataSize > fs.option.ConcurrentUploadLimit {
  83. glog.V(4).Infof("wait because inflight data %d > %d", inFlightDataSize, fs.option.ConcurrentUploadLimit)
  84. fs.inFlightDataLimitCond.Wait()
  85. inFlightDataSize = atomic.LoadInt64(&fs.inFlightDataSize)
  86. }
  87. fs.inFlightDataLimitCond.L.Unlock()
  88. atomic.AddInt64(&fs.inFlightDataSize, contentLength)
  89. defer func() {
  90. atomic.AddInt64(&fs.inFlightDataSize, -contentLength)
  91. fs.inFlightDataLimitCond.Signal()
  92. }()
  93. if r.Method == "PUT" {
  94. if _, ok := r.URL.Query()["tagging"]; ok {
  95. fs.PutTaggingHandler(w, r)
  96. } else {
  97. fs.PostHandler(w, r, contentLength)
  98. }
  99. } else { // method == "POST"
  100. fs.PostHandler(w, r, contentLength)
  101. }
  102. }
  103. }
  104. func (fs *FilerServer) readonlyFilerHandler(w http.ResponseWriter, r *http.Request) {
  105. start := time.Now()
  106. statusRecorder := stats.NewStatusResponseWriter(w)
  107. w = statusRecorder
  108. os.Stdout.WriteString("Request: " + r.Method + " " + r.URL.String() + "\n")
  109. origin := r.Header.Get("Origin")
  110. if origin != "" {
  111. if fs.option.AllowedOrigins == nil || len(fs.option.AllowedOrigins) == 0 || fs.option.AllowedOrigins[0] == "*" {
  112. origin = "*"
  113. } else {
  114. originFound := false
  115. for _, allowedOrigin := range fs.option.AllowedOrigins {
  116. if origin == allowedOrigin {
  117. originFound = true
  118. }
  119. }
  120. if !originFound {
  121. writeJsonError(w, r, http.StatusForbidden, errors.New("origin not allowed"))
  122. return
  123. }
  124. }
  125. w.Header().Set("Access-Control-Allow-Origin", origin)
  126. w.Header().Set("Access-Control-Allow-Headers", "OPTIONS, GET, HEAD")
  127. w.Header().Set("Access-Control-Allow-Credentials", "true")
  128. }
  129. defer func() {
  130. stats.FilerRequestCounter.WithLabelValues(r.Method, strconv.Itoa(statusRecorder.Status)).Inc()
  131. stats.FilerRequestHistogram.WithLabelValues(r.Method).Observe(time.Since(start).Seconds())
  132. }()
  133. // We handle OPTIONS first because it never should be authenticated
  134. if r.Method == "OPTIONS" {
  135. OptionsHandler(w, r, true)
  136. return
  137. }
  138. if !fs.maybeCheckJwtAuthorization(r, false) {
  139. writeJsonError(w, r, http.StatusUnauthorized, errors.New("wrong jwt"))
  140. return
  141. }
  142. w.Header().Set("Server", "SeaweedFS Filer "+util.VERSION)
  143. switch r.Method {
  144. case "GET":
  145. fs.GetOrHeadHandler(w, r)
  146. case "HEAD":
  147. fs.GetOrHeadHandler(w, r)
  148. }
  149. }
  150. func OptionsHandler(w http.ResponseWriter, r *http.Request, isReadOnly bool) {
  151. if isReadOnly {
  152. w.Header().Set("Access-Control-Allow-Methods", "GET, OPTIONS")
  153. } else {
  154. w.Header().Set("Access-Control-Allow-Methods", "PUT, POST, GET, DELETE, OPTIONS")
  155. w.Header().Set("Access-Control-Expose-Headers", "*")
  156. }
  157. w.Header().Set("Access-Control-Allow-Headers", "*")
  158. w.Header().Set("Access-Control-Allow-Credentials", "true")
  159. }
  160. // maybeCheckJwtAuthorization returns true if access should be granted, false if it should be denied
  161. func (fs *FilerServer) maybeCheckJwtAuthorization(r *http.Request, isWrite bool) bool {
  162. var signingKey security.SigningKey
  163. if isWrite {
  164. if len(fs.filerGuard.SigningKey) == 0 {
  165. return true
  166. } else {
  167. signingKey = fs.filerGuard.SigningKey
  168. }
  169. } else {
  170. if len(fs.filerGuard.ReadSigningKey) == 0 {
  171. return true
  172. } else {
  173. signingKey = fs.filerGuard.ReadSigningKey
  174. }
  175. }
  176. tokenStr := security.GetJwt(r)
  177. if tokenStr == "" {
  178. glog.V(1).Infof("missing jwt from %s", r.RemoteAddr)
  179. return false
  180. }
  181. token, err := security.DecodeJwt(signingKey, tokenStr, &security.SeaweedFilerClaims{})
  182. if err != nil {
  183. glog.V(1).Infof("jwt verification error from %s: %v", r.RemoteAddr, err)
  184. return false
  185. }
  186. if !token.Valid {
  187. glog.V(1).Infof("jwt invalid from %s: %v", r.RemoteAddr, tokenStr)
  188. return false
  189. } else {
  190. return true
  191. }
  192. }
  193. func (fs *FilerServer) filerHealthzHandler(w http.ResponseWriter, r *http.Request) {
  194. w.Header().Set("Server", "SeaweedFS Filer "+util.VERSION)
  195. w.WriteHeader(http.StatusOK)
  196. }