Kegsay
1591d1b0cb
|
8 years ago | |
---|---|---|
hooks | 9 years ago | |
src/github.com/matrix-org/go-neb | 8 years ago | |
vendor | 8 years ago | |
.gitignore | 9 years ago | |
README.md | 8 years ago |
README.md
Go-NEB
Go-NEB is a Matrix bot written in Go. It is the successor to Matrix-NEB, the original Matrix bot written in Python.
Table of Contents
Quick Start
Clone and run (Requires Go 1.5+ and GB):
gb build github.com/matrix-org/go-neb
BIND_ADDRESS=:4050 DATABASE_TYPE=sqlite3 DATABASE_URL=go-neb.db BASE_URL=http://localhost:4050 bin/go-neb
Get a Matrix user ID and access token and give it to Go-NEB:
curl -X POST localhost:4050/admin/configureClient --data-binary '{
"UserID": "@goneb:localhost",
"HomeserverURL": "http://localhost:8008",
"AccessToken": "<access_token>",
"Sync": true,
"AutoJoinRooms": true,
"DisplayName": "My Bot"
}'
Tell it what service to run:
curl -X POST localhost:4050/admin/configureService --data-binary '{
"Type": "echo",
"Id": "myserviceid",
"UserID": "@goneb:localhost",
"Config": {}
}'
Invite the bot user into a Matrix room and type !echo hello world
. It will reply with hello world
.
Features
Github
- Login with OAuth2.
- Ability to create Github issues on any project.
- Ability to track updates (add webhooks) to projects. This includes new issues, pull requests as well as commits.
- Ability to expand issues when mentioned as
foo/bar#1234
. - Ability to assign a "default repository" for a Matrix room to allow
#1234
to automatically expand, as well as shorter issue creation command syntax.
JIRA
- Login with OAuth1.
- Ability to create JIRA issues on a project.
- Ability to expand JIRA issues when mentioned as
FOO-1234
.
Giphy
- Ability to query Giphy's "text-to-gif" engine.
Installing
Go-NEB is built using Go 1.5+ and GB. Once you have installed Go, run the following commands:
# Install gb
go get github.com/constabulary/gb/...
# Clone the go-neb repository
git clone https://github.com/matrix-org/go-neb
cd go-neb
# Build go-neb
gb build github.com/matrix-org/go-neb
Running
Go-NEB uses environment variables to configure its SQLite database and bind address. To run Go-NEB, run the following command:
BIND_ADDRESS=:4050 DATABASE_TYPE=sqlite3 DATABASE_URL=go-neb.db BASE_URL=https://public.facing.endpoint bin/go-neb
BIND_ADDRESS
is the port to listen on.DATABASE_TYPE
MUST be "sqlite3". No other type is supported.DATABASE_URL
is where to find the database file. One will be created if it does not exist.BASE_URL
should be the public-facing endpoint that sites like Github can send webhooks to.
Go-NEB needs to be "configured" with clients and services before it will do anything useful.
Configuring Clients
Go-NEB needs to connect as a matrix user to receive messages. Go-NEB can listen for messages as multiple matrix users. The users are configured using an HTTP API and the config is stored in the database. To create a user:
curl -X POST localhost:4050/admin/configureClient --data-binary '{
"UserID": "@goneb:localhost:8448",
"HomeserverURL": "http://localhost:8008",
"AccessToken": "<access_token>",
"Sync": true,
"AutoJoinRooms": true,
"DisplayName": "My Bot"
}'
UserID
is the complete user ID of the client to connect as. The user MUST already exist.HomeserverURL
is the complete Homeserver URL for the given user ID.AccessToken
is the user's access token.Sync
, iftrue
, will start a/sync
stream so this client will receive incoming messages. This is required for services which need a live stream to the server (e.g. to respond to!commands
and expand issues). It is not required for services which do not respond to Matrix users (e.g. webhook notifications).AutoJoinRooms
, iftrue
, will automatically join rooms when an invite is received. This option is only valid whenSync: true
.DisplayName
, if set, will set the given user's profile display name to the string given.
Go-NEB will respond with the previous configuration for this client, if one exists, as well as echo back the complete configuration for the client:
{
"OldClient": {},
"NewClient": {
"UserID": "@goneb:localhost:8448",
"HomeserverURL": "http://localhost:8008",
"AccessToken": "<access_token>",
"Sync": true,
"AutoJoinRooms": true,
"DisplayName": "My Bot"
}
}
Configuring Services
Services contain all the useful functionality in Go-NEB. They require a client to operate. Services are configured using an HTTP API and the config is stored in the database. Services use one of the matrix users configured on Go-NEB to send/receive matrix messages.
Every service MUST have the following fields:
Type
: The type of service. This determines which code is executed.Id
: An arbitrary string which you can use to identify this service.UserID
: A user ID of a client which has been previously configured on Go-NEB. If this user does not exist, an error will be returned.Config
: A JSON object. The contents of this object depends on the service.
The information about a Service can be retrieved based on their Id
like so:
curl -X POST localhost:4050/admin/getService --data-binary '{
"Id": "myserviceid"
}'
This will return:
# HTTP 200 OK
{
"Type": "echo",
"Id": "myserviceid",
"UserID": "@goneb:localhost:8448",
"Config": {}
}
If the service is not found, this will return:
# HTTP 404 Not Found
{ "message": "Service not found" }
Echo Service
The simplest service. This will echo back any !echo
command. To configure one:
curl -X POST localhost:4050/admin/configureService --data-binary '{
"Type": "echo",
"Id": "myserviceid",
"UserID": "@goneb:localhost:8448",
"Config": {
}
}'
Then invite @goneb:localhost:8448
to any Matrix room and it will automatically join (if the client was configured to do so). Then try typing !echo hello world
and the bot will respond with hello world
.
Github Service
Before you can set up a Github Service, you need to set up a Github Realm.
This service will add the following command:
!github create owner/repo "Some title" "Some description"
JIRA Service
Giphy Service
A simple service that adds the ability to use the !giphy
command. To configure one:
curl -X POST localhost:4050/admin/configureService --data-binary '{
"Type": "giphy",
"Id": "giphyid",
"UserID": "@goneb:localhost",
"Config": {
"APIKey": "YOUR_API_KEY"
}
}'
Then invite the user into a room and type !giphy food
and it will respond with a GIF.
Configuring Realms
Realms are how Go-NEB authenticates users on third-party websites. Every realm MUST have the following fields:
ID
: An arbitrary string you can use to remember what the realm is.Type
: The type of realm. This determines what code gets executed.Config
: A JSON object. The contents depends on the realmType
.
They are configured like so:
curl -X POST localhost:4050/admin/configureAuthRealm --data-binary '{
"ID": "some_arbitrary_string",
"Type": "some_realm_type",
"Config": {
...
}
}'
Github Realm
This has the Type
of github
. The Config
object looks like:
{
"ClientSecret": "YOUR_CLIENT_SECRET",
"ClientID": "YOUR_CLIENT_ID",
"StarterLink": "https://example.com/requestGithubOAuthToken"
}
ClientSecret
: Your Github application client secretClientID
: Your Github application client IDStarterLink
: Optional. If supplied,!github
commands will return this link whenever someone is prompted to login to Github.
For example:
curl -X POST localhost:4050/admin/configureAuthRealm --data-binary '{
"ID": "mygithubrealm",
"Type": "github",
"Config": {
"ClientSecret": "YOUR_CLIENT_SECRET",
"ClientID": "YOUR_CLIENT_ID"
}
}'
JIRA Realm
This has the Type
of jira
. The Config
object looks like:
{
"JIRAEndpoint": "matrix.org/jira/",
"ConsumerName": "goneb",
"ConsumerKey": "goneb",
"ConsumerSecret": "random_long_string",
"PrivateKeyPEM": "-----BEGIN RSA PRIVATE KEY-----\r\nMIIEowIBAAKCAQEA39UhbOvQHEkBP9fGnhU+eSObTWBDGWygVYzbcONOlqEOTJUN\r\n8gmnellWqJO45S4jB1vLLnuXiHqEWnmaShIvbUem3QnDDqghu0gfqXHMlQr5R8ZP\r\norTt1F2idWy1wk5rVXeLKSG7uriYhDVOVS69WuefoW5v55b5YZV283v2jROjxHuj\r\ngAsJA7k6tvpYiSXApUl6YHmECfBoiwG9bwItkHwhZ\/fG9i4H8\/aOyr3WlaWbVeKX\r\n+m38lmYZvzQFRAk5ab1vzCGz4cyc\r\nTk2qmZpcjHRd1ijcOkgC23KF8lHWF5Zx0tySR+DWL1JeGm8NJxKMRJZuE8MIkJYF\r\nryE7kjspNItk6npkA3\/A4PWwElhddI4JpiuK+29mMNipRcYYy9e0vH\/igejv7ayd\r\nPLCRMQKBgBDSNWlZT0nNd2DXVqTW9p+MG72VKhDgmEwFB1acOw0lpu1XE8R1wmwG\r\nZRl\/xzri3LOW2Gpc77xu6fs3NIkzQw3v1ifYhX3OrVsCIRBbDjPQI3yYjkhGx24s\r\nVhhZ5S\/TkGk3Kw59bDC6KGqAuQAwX9req2l1NiuNaPU9rE7tf6Bk\r\n-----END RSA PRIVATE KEY-----",
"StarterLink": "https://example.com/requestJIRAOAuthToken"
}
JIRAEndpoint
: The base URL of the JIRA installation you wish to talk to.ConsumerName
: The desired "Consumer Name" field of the "Application Links" admin page on JIRA. Generally this is the name of the service. Users will need to enter this string into their JIRA admin web form.ConsumerKey
: The desired "Consumer Key" field of the "Application Links" admin page on JIRA. Generally this is the name of the service. Users will need to enter this string into their JIRA admin web form.ConsumerSecret
: The desired "Consumer Secret" field of the "Application Links" admin page on JIRA. This should be a random long string. Users will need to enter this string into their JIRA admin web form.PrivateKeyPEM
: A string which contains the private key for performing OAuth 1.0 requests. This MUST be in PEM format. It must NOT have a password. Go-NEB will convert this into a public key in PEM format and return this to users. Users will need to enter the public key into their JIRA admin web form.StarterLink
: Optional. If supplied,!jira
commands will return this link whenever someone is prompted to login to JIRA.
To generate a private key PEM: (JIRA does not support bit lengths >2048)
openssl genrsa -out privkey.pem 2048
cat privkey.pem
For example:
curl -X POST localhost:4050/admin/configureAuthRealm --data-binary '{
"ID": "jirarealm",
"Type": "jira",
"Config": {
"JIRAEndpoint": "matrix.org/jira/",
"ConsumerName": "goneb",
"ConsumerKey": "goneb",
"ConsumerSecret": "random_long_string",
"PrivateKeyPEM": "-----BEGIN RSA PRIVATE KEY-----\r\nMIIEowIBAAKCAQEA39UhbOvQHEkBP9fGnhU+eSObTWBDGWygVYzbcONOlqEOTJUN\r\n8gmnellWqJO45S4jB1vLLnuXiHqEWnmaShIvbUem3QnDDqghu0gfqXHMlQr5R8ZP\r\norTt1F2idWy1wk5rVXeLKSG7uriYhDVOVS69WuefoW5v55b5YZV283v2jROjxHuj\r\ngAsJA7k6tvpYiSXApUl6YHmECfBoiwG9bwItkHwhZ\/fG9i4H8\/aOyr3WlaWbVeKX\r\n+m38lmYZvzQFRAk5ab1vzCGz4cyc\r\nTk2qmZpcjHRd1ijcOkgC23KF8lHWF5Zx0tySR+DWL1JeGm8NJxKMRJZuE8MIkJYF\r\nryE7kjspNItk6npkA3\/A4PWwElhddI4JpiuK+29mMNipRcYYy9e0vH\/igejv7ayd\r\nPLCRMQKBgBDSNWlZT0nNd2DXVqTW9p+MG72VKhDgmEwFB1acOw0lpu1XE8R1wmwG\r\nZRl\/xzri3LOW2Gpc77xu6fs3NIkzQw3v1ifYhX3OrVsCIRBbDjPQI3yYjkhGx24s\r\nVhhZ5S\/TkGk3Kw59bDC6KGqAuQAwX9req2l1NiuNaPU9rE7tf6Bk\r\n-----END RSA PRIVATE KEY-----"
}
}'
Developing
OLD STUFF BELOW
Building go-neb
Go-neb is built using gb
(https://getgb.io/). To build go-neb:
# Install gb
go get github.com/constabulary/gb/...
# Clone the go-neb repository
git clone https://github.com/matrix-org/go-neb
cd go-neb
# Build go-neb
gb build github.com/matrix-org/go-neb
Running go-neb
Go-neb uses environment variables to configure its database and bind address. To run go-neb:
BIND_ADDRESS=:4050 DATABASE_TYPE=sqlite3 DATABASE_URL=go-neb.db BASE_URL=https://public.facing.endpoint bin/go-neb
Go-neb needs to connect as a matrix user to receive messages. Go-neb can listen for messages as multiple matrix users. The users are configured using an HTTP API and the config is stored in the database. Go-neb will automatically start syncing matrix messages when the user is configured. To create a user:
curl -X POST localhost:4050/admin/configureClient --data-binary '{
"UserID": "@goneb:localhost:8448",
"HomeserverURL": "http://localhost:8008",
"AccessToken": "<access_token>"
}'
{
"OldClient": {},
"NewClient": {
"UserID": "@goneb:localhost:8448",
"HomeserverURL": "http://localhost:8008",
"AccessToken": "<access_token>"
}
}
Services in go-neb listen for messages in particular rooms using a given matrix user. Services are configured using an HTTP API and the config is stored in the database. Services use one of the matrix users configured on go-neb to receive matrix messages. Each service is configured to listen for messages in a set of rooms. Go-neb will automatically join the service to its rooms when it is configured. To start an echo service:
curl -X POST localhost:4050/admin/configureService --data-binary '{
"Type": "echo",
"Id": "myserviceid",
"UserID": "@goneb:localhost:8448",
"Config": {
}
}'
{
"Type": "echo",
"Id": "myserviceid",
"UserID": "@goneb:localhost:8448",
"OldConfig": {},
"NewConfig": {}
}
To retrieve an existing Service:
curl -X POST localhost:4050/admin/getService --data-binary '{
"Id": "myserviceid"
}'
{
"Type": "echo",
"Id": "myserviceid",
"UserID": "@goneb:localhost:8448",
"Config": {}
}
Go-neb has a heartbeat listener that returns 200 OK so that load balancers can check that the server is still running.
curl -X GET localhost:4050/test
{}
Architecture
HOMESERVER
|
+=============================================================+
| | Go-NEB |
| +---------+ |
| | Clients | |
| +---------+ |
| | |
| +---------+ +------------+ +--------------+ |
| | Service |-------| Auth Realm |------| Auth Session |-+ |
| +---------+ +------------+ +--------------+ | |
| ^ ^ +---------------+ |
| | | |
+=============================================================+
| |
WEBHOOK REDIRECT
REQUEST REQUEST
Clients = A thing which can talk to homeservers and listen for events.
Service = An individual bot, configured by a user.
Auth Realm = A place where a user can authenticate with.
Auth Session = An individual authentication session
Some AuthRealms
support "Starter Links". These are HTTP URLs which knowledgeable clients should use to start the auth process. They are commonly returned as metadata to !commands
.
These links require the client to prove that they own a given user ID by appending a token
to the Starter Link. This token will be used to verify the client's identity by making an
Open ID request to the user's Homeserver via federation.
Starting a Github Service
Register a Github realm
This API allows for an optional StarterLink
value.
curl -X POST localhost:4050/admin/configureAuthRealm --data-binary '{
"ID": "mygithubrealm",
"Type": "github",
"Config": {
"ClientSecret": "YOUR_CLIENT_SECRET",
"ClientID": "YOUR_CLIENT_ID",
"StarterLink": "https://example.com/requestGithubOAuthToken"
}
}'
Returns:
{
"ID":"mygithubrealm",
"Type":"github",
"OldConfig":null,
"NewConfig":{
"ClientSecret":"YOUR_CLIENT_SECRET",
"ClientID":"YOUR_CLIENT_ID",
"StarterLink": "https://example.com/requestGithubOAuthToken"
}
}
Make a request for Github Auth
curl -X POST localhost:4050/admin/requestAuthSession --data-binary '{
"RealmID": "mygithubrealm",
"UserID": "@your_user_id:localhost",
"Config": {
"RedirectURL": "https://optional-url.com/to/redirect/to/after/auth"
}
}'
Returns:
{
"URL":"https://github.com/login/oauth/authorize?client_id=$ID\u0026client_secret=$SECRET\u0026redirect_uri=$REDIRECT_BASE_URI%2Frealms%2Fredirects%2Fmygithubrealm\u0026state=$RANDOM_STRING"
}
Follow this link and grant access for NEB to act on your behalf.
Create a github bot
curl -X POST localhost:4050/admin/configureService --data-binary '{
"Type": "github",
"Id": "mygithubserviceid",
"UserID": "@goneb:localhost",
"Config": {
"RealmID": "mygithubrealm",
"ClientUserID": "@example:localhost",
"HandleCommands": true,
"HandleExpansions": true,
"Rooms": {
"!EmwxeXCVubhskuWvaw:localhost": {
"Repos": {
"owner/repo": {
"Events": ["push","issues"]
}
}
}
}
}
}'
This request will make BotUserID
join the Rooms
specified and create webhooks for the owner/repo
projects given.
Starting a JIRA Service
Register a JIRA realm
Generate an RSA private key: (JIRA does not support key sizes >2048 bits)
openssl genrsa -out privkey.pem 2048
cat privkey.pem
This API allows for an optional StarterLink
value. Create the realm:
curl -X POST localhost:4050/admin/configureAuthRealm --data-binary '{
"ID": "jirarealm",
"Type": "jira",
"Config": {
"JIRAEndpoint": "matrix.org/jira/",
"StarterLink": "https://example.com/requestJIRAOAuthToken",
"ConsumerName": "goneb",
"ConsumerKey": "goneb",
"ConsumerSecret": "random_long_string",
"PrivateKeyPEM": "-----BEGIN RSA PRIVATE KEY-----\r\nMIIEowIBAAKCAQEA39UhbOvQHEkBP9fGnhU+eSObTWBDGWygVYzbcONOlqEOTJUN\r\n8gmnellWqJO45S4jB1vLLnuXiHqEWnmaShIvbUem3QnDDqghu0gfqXHMlQr5R8ZP\r\norTt1F2idWy1wk5rVXeLKSG7uriYhDVOVS69WuefoW5v55b5YZV283v2jROjxHuj\r\ngAsJA7k6tvpYiSXApUl6YHmECfBoiwG9bwItkHwhZ\/fG9i4H8\/aOyr3WlaWbVeKX\r\n+m38lmYZvzQFRAk5ab1vzCGz4cyc\r\nTk2qmZpcjHRd1ijcOkgC23KF8lHWF5Zx0tySR+DWL1JeGm8NJxKMRJZuE8MIkJYF\r\nryE7kjspNItk6npkA3\/A4PWwElhddI4JpiuK+29mMNipRcYYy9e0vH\/igejv7ayd\r\nPLCRMQKBgBDSNWlZT0nNd2DXVqTW9p+MG72VKhDgmEwFB1acOw0lpu1XE8R1wmwG\r\nZRl\/xzri3LOW2Gpc77xu6fs3NIkzQw3v1ifYhX3OrVsCIRBbDjPQI3yYjkhGx24s\r\nVhhZ5S\/TkGk3Kw59bDC6KGqAuQAwX9req2l1NiuNaPU9rE7tf6Bk\r\n-----END RSA PRIVATE KEY-----"
}
}'
The following keys will be modified/added:
JIRAEndpoint
in canonicalised form.Server
andVersion
keys which are purely informational for the caller.PublicKeyPEM
which the caller needs a human to insert into the JIRA Application Links web form.
Returns:
{
"ID": "jirarealm",
"Type": "jira",
"OldConfig": null,
"NewConfig": {
"JIRAEndpoint": "https://matrix.org/jira/",
"StarterLink": "https://example.com/requestJIRAOAuthToken",
"Server": "Matrix.org",
"Version": "6.3.5a",
"ConsumerName": "goneb",
"ConsumerKey": "goneb",
"ConsumerSecret": "random_long_string",
"PublicKeyPEM": "-----BEGIN PUBLIC KEY-----\nMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA39UhbOvQHEkBP9fGnhU+\neSObTWBDGWygVYzbcONOlqEOTJUN8gmnellWqJO45S4jB1vLLnuXiHqEWnmaShIv\nbUem3QnDDqghu0gfqXHMlQr5R8ZPorTt1F2idWy1wk5rVXeLKSG7uriYhDVOVS69\nWuefoW5v55b5YZV283v2jROjxHujgAsJA7k6tvpYiSXApUl6YHmECfBoiwG9bwIt\nkHwhZ/fG9i4H8/aOyr3WlaWbVeKX+m38lmYZvzQFRd7UPU7DuO6Aiqj7RxrbAvqq\ndPeoAvo6+V0TRPZ8YzKp2yQmDcGH69IbuKJ2BG1Qx8znZAvghKQ6P9Im+M4c7j9i\ndwIDAQAB\n-----END PUBLIC KEY-----\n",
"PrivateKeyPEM": "-----BEGIN RSA PRIVATE KEY-----\r\nMIIEowIBAAKCAQEA39UhbOvQHEkBP9fGnhU+eSObTWBDGWygVYzbcONOlqEOTJUN\r\n8gmnellWqJO45S4jB1vLLnuXiHqEWnmaShIvbUem3QnDDqghu0gfqXHMlQr5R8ZP\r\norTt1F2idWy1wk5rVXeLKSG7uriYhDVOVS69WuefoW5v55b5YZV283v2jROjxHuj\r\ngAsJA7k6tvpYiSXApUl6YHmECfBoiwG9bwItkHwhZ/fG9i4H8/aOyr3WlaWbVeKX\r\n+m38lmYZvzQFRd7UPU7DuO6Aiqj7RxrbAvqqdPeoAvo6+V0TRPZ8YzKp2yQmDcGH\r\n69IbuKJ2BG1Qx8znZAvghKQ6P9Im+M4c7j9iMG72VKhDgmEwFB1acOw0lpu1XE8R1wmwG\r\nZRl/xzri3LOW2Gpc77xu6fs3NIkzQw3v1ifYhX3OrVsCIRBbDjPQI3yYjkhGx24s\r\nVhhZ5S/TkGk3Kw59bDC6KGqAuQAwX9req2l1NiuNaPU9rE7tf6Bk\r\n-----END RSA PRIVATE KEY-----"
}
}
The ConsumerKey
, ConsumerSecret
, ConsumerName
and PublicKeyPEM
must be manually inserted
into the "Application Links" section under JIRA Admin Settings by a JIRA admin on the target
JIRA installation. Once that is complete, users can OAuth on the target JIRA installation.
Make a request for JIRA Auth
curl -X POST localhost:4050/admin/requestAuthSession --data-binary '{
"RealmID": "jirarealm",
"UserID": "@example:localhost",
"Config": {
"RedirectURL": "https://optional-url.com/to/redirect/to/after/auth"
}
}'
Returns:
{
"URL":"https://jira.somewhere.com/plugins/servlet/oauth/authorize?oauth_token=7yeuierbgweguiegrTbOT"
}
Follow this link and grant access for NEB to act on your behalf.
Create a JIRA bot
curl -X POST localhost:4050/admin/configureService --data-binary '{
"Type": "jira",
"Id": "jid",
"UserID": "@goneb:localhost",
"Config": {
"ClientUserID": "@example:localhost",
"Rooms": {
"!EmwxeXCVubhskuWvaw:localhost": {
"Realms": {
"jira_realm_id": {
"Projects": {
"BOTS": {
"Expand": true,
"Track": true
}
}
}
}
}
}
}
}'
Starting a Giphy Service
Create a Giphy bot
curl -X POST localhost:4050/admin/configureService --data-binary '{
"Type": "giphy",
"Id": "giphyid",
"UserID": "@goneb:localhost",
"Config": {
"APIKey": "YOUR_API_KEY"
}
}'
Developing on go-neb.
There's a bunch more tools this project uses when developing in order to do things like linting. Some of them are bundled with go (fmt and vet) but some are not. You should install the ones which are not:
go get github.com/golang/lint/golint
go get github.com/fzipp/gocyclo
You can then install the pre-commit hook:
./hooks/install.sh
Viewing the API docs.
# Start a documentation server listening on :6060
GOPATH=$GOPATH:$(pwd) godoc -v -http=localhost:6060 &
# Open up the documentation for go-neb in a browser.
sensible-browser http://localhost/pkg/github.com/matrix-org/go-neb