You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

296 lines
13 KiB

8 years ago
8 years ago
8 years ago
8 years ago
8 years ago
8 years ago
8 years ago
8 years ago
8 years ago
8 years ago
8 years ago
8 years ago
  1. # Go-NEB
  2. Go-NEB is a [Matrix](https://matrix.org) bot written in Go. It is the successor to [Matrix-NEB](https://github.com/matrix-org/Matrix-NEB), the original Matrix bot written in Python.
  3. # Table of Contents
  4. * [Quick Start](#quick-start)
  5. * [Features](#features)
  6. * [Installing](#installing)
  7. * [Running](#running)
  8. * [Configuration file](#configuration-file)
  9. * [Configuring clients](#configuring-clients)
  10. * [Configuring services](#configuring-services)
  11. * [Echo Service](#echo-service)
  12. * [Github Service](#github-service)
  13. * [Github Webhook Service](#github-webhook-service)
  14. * [JIRA Service](#jira-service)
  15. * [Giphy Service](#giphy-service)
  16. * [Configuring realms](#configuring-realms)
  17. * [Github Realm](#github-realm)
  18. * [Github Authentication](#github-authentication)
  19. * [JIRA Realm](#jira-realm)
  20. * [Developing](#developing)
  21. * [Architecture](#architecture)
  22. * [API Docs](#viewing-the-api-docs)
  23. # Quick Start
  24. Clone and run (Requires Go 1.5+ and GB):
  25. ```bash
  26. gb build github.com/matrix-org/go-neb
  27. BIND_ADDRESS=:4050 DATABASE_TYPE=sqlite3 DATABASE_URL=go-neb.db?_busy_timeout=5000 BASE_URL=http://localhost:4050 bin/go-neb
  28. ```
  29. Get a Matrix user ID and access token and give it to Go-NEB:
  30. ```bash
  31. curl -X POST localhost:4050/admin/configureClient --data-binary '{
  32. "UserID": "@goneb:localhost",
  33. "HomeserverURL": "http://localhost:8008",
  34. "AccessToken": "<access_token>",
  35. "Sync": true,
  36. "AutoJoinRooms": true,
  37. "DisplayName": "My Bot"
  38. }'
  39. ```
  40. Tell it what service to run:
  41. ```bash
  42. curl -X POST localhost:4050/admin/configureService --data-binary '{
  43. "Type": "echo",
  44. "Id": "myserviceid",
  45. "UserID": "@goneb:localhost",
  46. "Config": {}
  47. }'
  48. ```
  49. Invite the bot user into a Matrix room and type `!echo hello world`. It will reply with `hello world`.
  50. ## Features
  51. ### Github
  52. - Login with OAuth2.
  53. - Ability to create Github issues on any project.
  54. - Ability to track updates (add webhooks) to projects. This includes new issues, pull requests as well as commits.
  55. - Ability to expand issues when mentioned as `foo/bar#1234`.
  56. - Ability to assign a "default repository" for a Matrix room to allow `#1234` to automatically expand, as well as shorter issue creation command syntax.
  57. ### JIRA
  58. - Login with OAuth1.
  59. - Ability to create JIRA issues on a project.
  60. - Ability to expand JIRA issues when mentioned as `FOO-1234`.
  61. ### Giphy
  62. - Ability to query Giphy's "text-to-gif" engine.
  63. # Installing
  64. Go-NEB is built using Go 1.5+ and [GB](https://getgb.io/). Once you have installed Go, run the following commands:
  65. ```bash
  66. # Install gb
  67. go get github.com/constabulary/gb/...
  68. # Clone the go-neb repository
  69. git clone https://github.com/matrix-org/go-neb
  70. cd go-neb
  71. # Build go-neb
  72. gb build github.com/matrix-org/go-neb
  73. ```
  74. # Running
  75. Go-NEB uses environment variables to configure its SQLite database and bind address. To run Go-NEB, run the following command:
  76. ```bash
  77. BIND_ADDRESS=:4050 DATABASE_TYPE=sqlite3 DATABASE_URL=go-neb.db?_busy_timeout=5000 BASE_URL=https://public.facing.endpoint bin/go-neb
  78. ```
  79. - `BIND_ADDRESS` is the port to listen on.
  80. - `DATABASE_TYPE` MUST be "sqlite3". No other type is supported.
  81. - `DATABASE_URL` is where to find the database file. One will be created if it does not exist. It is a URL so parameters can be passed to it. We recommend setting `_busy_timeout=5000` to prevent sqlite3 "database is locked" errors.
  82. - `BASE_URL` should be the public-facing endpoint that sites like Github can send webhooks to.
  83. - `CONFIG_FILE` is the path to the configuration file to read from. This isn't included in the example above, so Go-NEB will operate in HTTP mode.
  84. Go-NEB needs to be "configured" with clients and services before it will do anything useful. It can be configured via a configuration file OR by an HTTP API.
  85. ## Configuration file
  86. If you run Go-NEB with a `CONFIG_FILE` environment variable, it will load that file and use it for services, clients, etc. There is a [sample configuration file](config.sample.yaml) which explains all the options. In most cases, these are *direct mappings* to the corresponding HTTP API.
  87. ## Configuring Clients
  88. Go-NEB needs to connect as a matrix user to receive messages. Go-NEB can listen for messages as multiple matrix users. The users are configured using an HTTP API and the config is stored in the database.
  89. - [HTTP API Docs](https://matrix-org.github.io/go-neb/pkg/github.com/matrix-org/go-neb/api/handlers/index.html#ConfigureClient.OnIncomingRequest)
  90. - [JSON Request Body Docs](https://matrix-org.github.io/go-neb/pkg/github.com/matrix-org/go-neb/api/index.html#ClientConfig)
  91. ## Configuring Services
  92. Services contain all the useful functionality in Go-NEB. They require a client to operate. Services are configured using an HTTP API and the config is stored in the database. Services use one of the matrix users configured on Go-NEB to send/receive matrix messages.
  93. Every service has an "ID", "type" and "user ID". Services may specify additional "config" keys: see the specific
  94. service you're interested in for the additional keys, if any.
  95. - [HTTP API Docs](https://matrix-org.github.io/go-neb/pkg/github.com/matrix-org/go-neb/api/handlers/index.html#ConfigureService.OnIncomingRequest)
  96. - [JSON Request Body Docs](https://matrix-org.github.io/go-neb/pkg/github.com/matrix-org/go-neb/api/index.html#ConfigureServiceRequest)
  97. List of services:
  98. - [Echo](https://matrix-org.github.io/go-neb/pkg/github.com/matrix-org/go-neb/services/echo/) - An example service
  99. - [Giphy](https://matrix-org.github.io/go-neb/pkg/github.com/matrix-org/go-neb/services/giphy/) - A GIF bot
  100. - [Github](https://matrix-org.github.io/go-neb/pkg/github.com/matrix-org/go-neb/services/github/) - A Github bot
  101. - [Github Webhook](https://matrix-org.github.io/go-neb/pkg/github.com/matrix-org/go-neb/services/github/index.html#WebhookService) - A Github notification bot
  102. - [Guggy](https://matrix-org.github.io/go-neb/pkg/github.com/matrix-org/go-neb/services/guggy/) - A GIF bot
  103. - [JIRA](https://matrix-org.github.io/go-neb/pkg/github.com/matrix-org/go-neb/services/jira/) - Integration with JIRA
  104. - [RSS Bot](https://matrix-org.github.io/go-neb/pkg/github.com/matrix-org/go-neb/services/rssbot/) - An Atom/RSS feed reader
  105. ## Configuring Realms
  106. Realms are how Go-NEB authenticates users on third-party websites.
  107. - [HTTP API Docs](https://matrix-org.github.io/go-neb/pkg/github.com/matrix-org/go-neb/api/handlers/index.html#ConfigureAuthRealm.OnIncomingRequest)
  108. - [JSON Request Body Docs](https://matrix-org.github.io/go-neb/pkg/github.com/matrix-org/go-neb/api/index.html#ConfigureAuthRealmRequest)
  109. ### Github Realm
  110. This has the `Type` of `github`. To set up this realm:
  111. ```bash
  112. curl -X POST localhost:4050/admin/configureAuthRealm --data-binary '{
  113. "ID": "mygithubrealm",
  114. "Type": "github",
  115. "Config": {
  116. "ClientSecret": "YOUR_CLIENT_SECRET",
  117. "ClientID": "YOUR_CLIENT_ID",
  118. "StarterLink": "https://example.com/requestGithubOAuthToken"
  119. }
  120. }'
  121. ```
  122. - `ClientSecret`: Your Github application client secret
  123. - `ClientID`: Your Github application client ID
  124. - `StarterLink`: Optional. If supplied, `!github` commands will return this link whenever someone is prompted to login to Github.
  125. #### Github authentication
  126. Once you have configured a Github realm, you can associate any Matrix user ID with any Github user. To do this:
  127. ```bash
  128. curl -X POST localhost:4050/admin/requestAuthSession --data-binary '{
  129. "RealmID": "mygithubrealm",
  130. "UserID": "@real_matrix_user:localhost",
  131. "Config": {
  132. "RedirectURL": "https://optional-url.com/to/redirect/to/after/auth"
  133. }
  134. }'
  135. ```
  136. - `UserID`: The Matrix user ID to associate with.
  137. - `RedirectURL`: Optional. The URL to redirect to after authentication.
  138. This request will return an OAuth URL:
  139. ```json
  140. {
  141. "URL": "https://github.com/login/oauth/authorize?client_id=abcdef&client_secret=acascacac...."
  142. }
  143. ```
  144. Follow this link to associate this user ID with this Github account. Once this is complete, Go-NEB will have an OAuth token for this user ID and will be able to create issues as their real Github account.
  145. To remove this session:
  146. ```bash
  147. curl -X POST localhost:4050/admin/removeAuthSession --data-binary '{
  148. "RealmID": "mygithubrealm",
  149. "UserID": "@real_matrix_user:localhost",
  150. "Config": {}
  151. }'
  152. ```
  153. ### JIRA Realm
  154. This has the `Type` of `jira`. To set up this realm:
  155. ```bash
  156. curl -X POST localhost:4050/admin/configureAuthRealm --data-binary '{
  157. "ID": "jirarealm",
  158. "Type": "jira",
  159. "Config": {
  160. "JIRAEndpoint": "matrix.org/jira/",
  161. "ConsumerName": "goneb",
  162. "ConsumerKey": "goneb",
  163. "ConsumerSecret": "random_long_string",
  164. "PrivateKeyPEM": "-----BEGIN RSA PRIVATE KEY-----\r\nMIIEowIBAAKCAQEA39UhbOvQHEkBP9fGnhU+eSObTWBDGWygVYzbcONOlqEOTJUN\r\n8gmnellWqJO45S4jB1vLLnuXiHqEWnmaShIvbUem3QnDDqghu0gfqXHMlQr5R8ZP\r\norTt1F2idWy1wk5rVXeLKSG7uriYhDVOVS69WuefoW5v55b5YZV283v2jROjxHuj\r\ngAsJA7k6tvpYiSXApUl6YHmECfBoiwG9bwItkHwhZ\/fG9i4H8\/aOyr3WlaWbVeKX\r\n+m38lmYZvzQFRAk5ab1vzCGz4cyc\r\nTk2qmZpcjHRd1ijcOkgC23KF8lHWF5Zx0tySR+DWL1JeGm8NJxKMRJZuE8MIkJYF\r\nryE7kjspNItk6npkA3\/A4PWwElhddI4JpiuK+29mMNipRcYYy9e0vH\/igejv7ayd\r\nPLCRMQKBgBDSNWlZT0nNd2DXVqTW9p+MG72VKhDgmEwFB1acOw0lpu1XE8R1wmwG\r\nZRl\/xzri3LOW2Gpc77xu6fs3NIkzQw3v1ifYhX3OrVsCIRBbDjPQI3yYjkhGx24s\r\nVhhZ5S\/TkGk3Kw59bDC6KGqAuQAwX9req2l1NiuNaPU9rE7tf6Bk\r\n-----END RSA PRIVATE KEY-----"
  165. }
  166. }'
  167. ```
  168. - `JIRAEndpoint`: The base URL of the JIRA installation you wish to talk to.
  169. - `ConsumerName`: The desired "Consumer Name" field of the "Application Links" admin page on JIRA. Generally this is the name of the service. Users will need to enter this string into their JIRA admin web form.
  170. - `ConsumerKey`: The desired "Consumer Key" field of the "Application Links" admin page on JIRA. Generally this is the name of the service. Users will need to enter this string into their JIRA admin web form.
  171. - `ConsumerSecret`: The desired "Consumer Secret" field of the "Application Links" admin page on JIRA. This should be a random long string. Users will need to enter this string into their JIRA admin web form.
  172. - `PrivateKeyPEM`: A string which contains the private key for performing OAuth 1.0 requests. This MUST be in PEM format. It must NOT have a password. Go-NEB will convert this into a **public** key in PEM format and return this to users. Users will need to enter the public key into their JIRA admin web form.
  173. - `StarterLink`: Optional. If supplied, `!jira` commands will return this link whenever someone is prompted to login to JIRA.
  174. To generate a private key PEM: (JIRA does not support bit lengths >2048)
  175. ```bash
  176. openssl genrsa -out privkey.pem 2048
  177. cat privkey.pem
  178. ```
  179. #### JIRA authentication
  180. ```
  181. curl -X POST localhost:4050/admin/requestAuthSession --data-binary '{
  182. "RealmID": "jirarealm",
  183. "UserID": "@example:localhost",
  184. "Config": {
  185. "RedirectURL": "https://optional-url.com/to/redirect/to/after/auth"
  186. }
  187. }'
  188. ```
  189. Returns:
  190. ```json
  191. {
  192. "URL":"https://jira.somewhere.com/plugins/servlet/oauth/authorize?oauth_token=7yeuierbgweguiegrTbOT"
  193. }
  194. ```
  195. # Developing
  196. There's a bunch more tools this project uses when developing in order to do
  197. things like linting. Some of them are bundled with go (fmt and vet) but some
  198. are not. You should install the ones which are not:
  199. ```bash
  200. go get github.com/golang/lint/golint
  201. go get github.com/fzipp/gocyclo
  202. ```
  203. You can then install the pre-commit hook:
  204. ```bash
  205. ./hooks/install.sh
  206. ```
  207. ## Architecture
  208. ```
  209. HOMESERVER
  210. |
  211. +=============================================================+
  212. | | Go-NEB |
  213. | +---------+ |
  214. | | Clients | |
  215. | +---------+ |
  216. | | |
  217. | +---------+ +------------+ +--------------+ |
  218. | | Service |-------| Auth Realm |------| Auth Session |-+ |
  219. | +---------+ +------------+ +--------------+ | |
  220. | ^ ^ +---------------+ |
  221. | | | |
  222. +=============================================================+
  223. | |
  224. WEBHOOK REDIRECT
  225. REQUEST REQUEST
  226. Clients = A thing which can talk to homeservers and listen for events. /configureClient makes these.
  227. Service = An individual bot, configured by a user. /configureService makes these.
  228. Auth Realm = A place where a user can authenticate with. /configureAuthRealm makes these.
  229. Auth Session = An individual authentication session /requestAuthSession makes these.
  230. ```
  231. ## Viewing the API docs
  232. The full docs can be found on [Github Pages](https://matrix-org.github.io/go-neb/pkg/github.com/matrix-org/go-neb). Alternatively, you can locally host the API docs:
  233. ```
  234. # Start a documentation server listening on :6060
  235. GOPATH=$GOPATH:$(pwd) godoc -v -http=localhost:6060 &
  236. # Open up the documentation for go-neb in a browser.
  237. sensible-browser http://localhost:6060/pkg/github.com/matrix-org/go-neb
  238. ```