18 KiB

Raw Blame History

Go-NEB

Go-NEB is a Matrix bot written in Go. It is the successor to Matrix-NEB, the original Matrix bot written in Python.

Quick Start
- Features
Installing
Running
- Configuring clients
- Configuring services
Developing

Quick Start

Clone and run (Requires Go 1.5+ and GB):

gb build github.com/matrix-org/go-neb
BIND_ADDRESS=:4050 DATABASE_TYPE=sqlite3 DATABASE_URL=go-neb.db BASE_URL=http://localhost:4050 bin/go-neb

Get a Matrix user ID and access token and give it to Go-NEB:

curl -X POST localhost:4050/admin/configureClient --data-binary '{
    "UserID": "@goneb:localhost",
    "HomeserverURL": "http://localhost:8008",
    "AccessToken": "<access_token>",
    "Sync": true,
    "AutoJoinRooms": true,
    "DisplayName": "My Bot"
}'

Tell it what service to run:

curl -X POST localhost:4050/admin/configureService --data-binary '{
    "Type": "echo",
    "Id": "myserviceid",
    "UserID": "@goneb:localhost",
    "Config": {}
}'

Invite the bot user into a Matrix room and type !echo hello world. It will reply with hello world.

Features

Github

Login with OAuth2.
Ability to create Github issues on any project.
Ability to track updates (add webhooks) to projects. This includes new issues, pull requests as well as commits.
Ability to expand issues when mentioned as foo/bar#1234.
Ability to assign a "default repository" for a Matrix room to allow #1234 to automatically expand, as well as shorter issue creation command syntax.

JIRA

Login with OAuth1.
Ability to create JIRA issues on a project.
Ability to expand JIRA issues when mentioned as FOO-1234.

Giphy

Ability to query Giphy's "text-to-gif" engine.

Installing

Go-NEB is built using Go 1.5+ and GB. Once you have installed Go, run the following commands:

# Install gb
go get github.com/constabulary/gb/...

# Clone the go-neb repository
git clone https://github.com/matrix-org/go-neb
cd go-neb

# Build go-neb
gb build github.com/matrix-org/go-neb

Running

Go-NEB uses environment variables to configure its SQLite database and bind address. To run Go-NEB, run the following command:

BIND_ADDRESS=:4050 DATABASE_TYPE=sqlite3 DATABASE_URL=go-neb.db BASE_URL=https://public.facing.endpoint bin/go-neb

BIND_ADDRESS is the port to listen on.
DATABASE_TYPE MUST be "sqlite3". No other type is supported.
DATABASE_URL is where to find the database file. One will be created if it does not exist.
BASE_URL should be the public-facing endpoint that sites like Github can send webhooks to.

Go-NEB needs to be "configured" with clients and services before it will do anything useful.

Configuring Clients

Go-NEB needs to connect as a matrix user to receive messages. Go-NEB can listen for messages as multiple matrix users. The users are configured using an HTTP API and the config is stored in the database. To create a user:

curl -X POST localhost:4050/admin/configureClient --data-binary '{
    "UserID": "@goneb:localhost:8448",
    "HomeserverURL": "http://localhost:8008",
    "AccessToken": "<access_token>",
    "Sync": true,
    "AutoJoinRooms": true,
    "DisplayName": "My Bot"
}'

UserID is the complete user ID of the client to connect as. The user MUST already exist.
HomeserverURL is the complete Homeserver URL for the given user ID.
AccessToken is the user's access token.
Sync, if true, will start a /sync stream so this client will receive incoming messages. This is required for services which need a live stream to the server (e.g. to respond to !commands and expand issues). It is not required for services which do not respond to Matrix users (e.g. webhook notifications).
AutoJoinRooms, if true, will automatically join rooms when an invite is received. This option is only valid when Sync: true.
DisplayName, if set, will set the given user's profile display name to the string given.

Go-NEB will respond with the previous configuration for this client, if one exists, as well as echo back the complete configuration for the client:

{
    "OldClient": {},
    "NewClient": {
        "UserID": "@goneb:localhost:8448",
        "HomeserverURL": "http://localhost:8008",
        "AccessToken": "<access_token>",
        "Sync": true,
        "AutoJoinRooms": true,
        "DisplayName": "My Bot"
    }
}

Configuring Services

Services contain all the useful functionality in Go-NEB. They require a client to operate. Services are configured using an HTTP API and the config is stored in the database. Services use one of the matrix users configured on Go-NEB to send/receive matrix messages.

Every service MUST have the following fields:

Type : The type of service. This determines which code is executed.
Id : An arbitrary string which you can use to identify this service.
UserID : A user ID of a client which has been previously configured on Go-NEB. If this user does not exist, an error will be returned.
Config : A JSON object. The contents of this object depends on the service.

The information about a Service can be retrieved based on their Id like so:

curl -X POST localhost:4050/admin/getService --data-binary '{
    "Id": "myserviceid"
}'

This will return:

# HTTP 200 OK
{
    "Type": "echo",
    "Id": "myserviceid",
    "UserID": "@goneb:localhost:8448",
    "Config": {}
}

If the service is not found, this will return:

# HTTP 404 Not Found
{ "message": "Service not found" }

Echo Service

The simplest service. This will echo back any !echo command. To configure one:

curl -X POST localhost:4050/admin/configureService --data-binary '{
    "Type": "echo",
    "Id": "myserviceid",
    "UserID": "@goneb:localhost:8448",
    "Config": {
    }
}'

Then invite @goneb:localhost:8448 to any Matrix room and it will automatically join (if the client was configured to do so). Then try typing !echo hello world and the bot will respond with hello world.

Github Service

JIRA Service

Giphy Service

Developing

OLD STUFF BELOW

Building go-neb

Go-neb is built using gb (https://getgb.io/). To build go-neb:

# Install gb
go get github.com/constabulary/gb/...

# Clone the go-neb repository
git clone https://github.com/matrix-org/go-neb
cd go-neb

# Build go-neb
gb build github.com/matrix-org/go-neb

Running go-neb

Go-neb uses environment variables to configure its database and bind address. To run go-neb:

BIND_ADDRESS=:4050 DATABASE_TYPE=sqlite3 DATABASE_URL=go-neb.db BASE_URL=https://public.facing.endpoint bin/go-neb

Go-neb needs to connect as a matrix user to receive messages. Go-neb can listen for messages as multiple matrix users. The users are configured using an HTTP API and the config is stored in the database. Go-neb will automatically start syncing matrix messages when the user is configured. To create a user:

curl -X POST localhost:4050/admin/configureClient --data-binary '{
    "UserID": "@goneb:localhost:8448",
    "HomeserverURL": "http://localhost:8008",
    "AccessToken": "<access_token>"
}'
{
    "OldClient": {},
    "NewClient": {
        "UserID": "@goneb:localhost:8448",
        "HomeserverURL": "http://localhost:8008",
        "AccessToken": "<access_token>"
    }
}

Services in go-neb listen for messages in particular rooms using a given matrix user. Services are configured using an HTTP API and the config is stored in the database. Services use one of the matrix users configured on go-neb to receive matrix messages. Each service is configured to listen for messages in a set of rooms. Go-neb will automatically join the service to its rooms when it is configured. To start an echo service:

curl -X POST localhost:4050/admin/configureService --data-binary '{
    "Type": "echo",
    "Id": "myserviceid",
    "UserID": "@goneb:localhost:8448",
    "Config": {
    }
}'
{
    "Type": "echo",
    "Id": "myserviceid",
    "UserID": "@goneb:localhost:8448",
    "OldConfig": {},
    "NewConfig": {}
}

To retrieve an existing Service:

curl -X POST localhost:4050/admin/getService --data-binary '{
    "Id": "myserviceid"
}'
{
    "Type": "echo",
    "Id": "myserviceid",
    "UserID": "@goneb:localhost:8448",
    "Config": {}
}

Go-neb has a heartbeat listener that returns 200 OK so that load balancers can check that the server is still running.

curl -X GET localhost:4050/test

{}

Architecture


   HOMESERVER
       |
+=============================================================+
|      |                 Go-NEB                               |
| +---------+                                                 |
| | Clients |                                                 |
| +---------+                                                 |
|      |                                                      |
| +---------+       +------------+      +--------------+      |
| | Service |-------| Auth Realm |------| Auth Session |-+    |
| +---------+       +------------+      +--------------+ |    |
|     ^                   ^              +---------------+    |
|     |                   |                                   |
+=============================================================+
      |                   |                   
    WEBHOOK            REDIRECT
    REQUEST            REQUEST
    
    
Clients      = A thing which can talk to homeservers and listen for events.
Service      = An individual bot, configured by a user.
Auth Realm   = A place where a user can authenticate with.
Auth Session = An individual authentication session

Some AuthRealms support "Starter Links". These are HTTP URLs which knowledgeable clients should use to start the auth process. They are commonly returned as metadata to !commands. These links require the client to prove that they own a given user ID by appending a token to the Starter Link. This token will be used to verify the client's identity by making an Open ID request to the user's Homeserver via federation.

Starting a Github Service

Register a Github realm

This API allows for an optional StarterLink value.

curl -X POST localhost:4050/admin/configureAuthRealm --data-binary '{
    "ID": "mygithubrealm",
    "Type": "github",
    "Config": {
        "ClientSecret": "YOUR_CLIENT_SECRET",
        "ClientID": "YOUR_CLIENT_ID",
        "StarterLink": "https://example.com/requestGithubOAuthToken"
    }
}'

Returns:

{
  "ID":"mygithubrealm",
  "Type":"github",
  "OldConfig":null,
  "NewConfig":{
    "ClientSecret":"YOUR_CLIENT_SECRET",
    "ClientID":"YOUR_CLIENT_ID",
    "StarterLink": "https://example.com/requestGithubOAuthToken"
  }
}

Make a request for Github Auth

curl -X POST localhost:4050/admin/requestAuthSession --data-binary '{
    "RealmID": "mygithubrealm",
    "UserID": "@your_user_id:localhost",
    "Config": {
        "RedirectURL": "https://optional-url.com/to/redirect/to/after/auth"
    }
}'

Returns:

{
  "URL":"https://github.com/login/oauth/authorize?client_id=$ID\u0026client_secret=$SECRET\u0026redirect_uri=$REDIRECT_BASE_URI%2Frealms%2Fredirects%2Fmygithubrealm\u0026state=$RANDOM_STRING"
}

Follow this link and grant access for NEB to act on your behalf.

Create a github bot

curl -X POST localhost:4050/admin/configureService --data-binary '{
    "Type": "github",
    "Id": "mygithubserviceid",
    "UserID": "@goneb:localhost",
    "Config": {
    	"RealmID": "mygithubrealm",
        "ClientUserID": "@example:localhost",
        "HandleCommands": true,
        "HandleExpansions": true,
        "Rooms": {
        	"!EmwxeXCVubhskuWvaw:localhost": {
        		"Repos": {
        			"owner/repo": {
        				"Events": ["push","issues"]
        			}
        		}
        	}
        }
    }
}'

This request will make BotUserID join the Rooms specified and create webhooks for the owner/repo projects given.

Starting a JIRA Service

Register a JIRA realm

Generate an RSA private key: (JIRA does not support key sizes >2048 bits)

openssl genrsa -out privkey.pem 2048
cat privkey.pem

This API allows for an optional StarterLink value. Create the realm:

curl -X POST localhost:4050/admin/configureAuthRealm --data-binary '{
    "ID": "jirarealm",
    "Type": "jira",
    "Config": {
        "JIRAEndpoint": "matrix.org/jira/",
        "StarterLink": "https://example.com/requestJIRAOAuthToken",
        "ConsumerName": "goneb",
        "ConsumerKey": "goneb",
        "ConsumerSecret": "random_long_string",
        "PrivateKeyPEM": "-----BEGIN RSA PRIVATE KEY-----\r\nMIIEowIBAAKCAQEA39UhbOvQHEkBP9fGnhU+eSObTWBDGWygVYzbcONOlqEOTJUN\r\n8gmnellWqJO45S4jB1vLLnuXiHqEWnmaShIvbUem3QnDDqghu0gfqXHMlQr5R8ZP\r\norTt1F2idWy1wk5rVXeLKSG7uriYhDVOVS69WuefoW5v55b5YZV283v2jROjxHuj\r\ngAsJA7k6tvpYiSXApUl6YHmECfBoiwG9bwItkHwhZ\/fG9i4H8\/aOyr3WlaWbVeKX\r\n+m38lmYZvzQFRAk5ab1vzCGz4cyc\r\nTk2qmZpcjHRd1ijcOkgC23KF8lHWF5Zx0tySR+DWL1JeGm8NJxKMRJZuE8MIkJYF\r\nryE7kjspNItk6npkA3\/A4PWwElhddI4JpiuK+29mMNipRcYYy9e0vH\/igejv7ayd\r\nPLCRMQKBgBDSNWlZT0nNd2DXVqTW9p+MG72VKhDgmEwFB1acOw0lpu1XE8R1wmwG\r\nZRl\/xzri3LOW2Gpc77xu6fs3NIkzQw3v1ifYhX3OrVsCIRBbDjPQI3yYjkhGx24s\r\nVhhZ5S\/TkGk3Kw59bDC6KGqAuQAwX9req2l1NiuNaPU9rE7tf6Bk\r\n-----END RSA PRIVATE KEY-----"
    }
}'

The following keys will be modified/added:

JIRAEndpoint in canonicalised form.
Server and Version keys which are purely informational for the caller.
PublicKeyPEM which the caller needs a human to insert into the JIRA Application Links web form.

Returns:

{
    "ID": "jirarealm",
    "Type": "jira",
    "OldConfig": null,
    "NewConfig": {
        "JIRAEndpoint": "https://matrix.org/jira/",
        "StarterLink": "https://example.com/requestJIRAOAuthToken",
        "Server": "Matrix.org",
        "Version": "6.3.5a",
        "ConsumerName": "goneb",
        "ConsumerKey": "goneb",
        "ConsumerSecret": "random_long_string",
        "PublicKeyPEM": "-----BEGIN PUBLIC KEY-----\nMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA39UhbOvQHEkBP9fGnhU+\neSObTWBDGWygVYzbcONOlqEOTJUN8gmnellWqJO45S4jB1vLLnuXiHqEWnmaShIv\nbUem3QnDDqghu0gfqXHMlQr5R8ZPorTt1F2idWy1wk5rVXeLKSG7uriYhDVOVS69\nWuefoW5v55b5YZV283v2jROjxHujgAsJA7k6tvpYiSXApUl6YHmECfBoiwG9bwIt\nkHwhZ/fG9i4H8/aOyr3WlaWbVeKX+m38lmYZvzQFRd7UPU7DuO6Aiqj7RxrbAvqq\ndPeoAvo6+V0TRPZ8YzKp2yQmDcGH69IbuKJ2BG1Qx8znZAvghKQ6P9Im+M4c7j9i\ndwIDAQAB\n-----END PUBLIC KEY-----\n",
        "PrivateKeyPEM": "-----BEGIN RSA PRIVATE KEY-----\r\nMIIEowIBAAKCAQEA39UhbOvQHEkBP9fGnhU+eSObTWBDGWygVYzbcONOlqEOTJUN\r\n8gmnellWqJO45S4jB1vLLnuXiHqEWnmaShIvbUem3QnDDqghu0gfqXHMlQr5R8ZP\r\norTt1F2idWy1wk5rVXeLKSG7uriYhDVOVS69WuefoW5v55b5YZV283v2jROjxHuj\r\ngAsJA7k6tvpYiSXApUl6YHmECfBoiwG9bwItkHwhZ/fG9i4H8/aOyr3WlaWbVeKX\r\n+m38lmYZvzQFRd7UPU7DuO6Aiqj7RxrbAvqqdPeoAvo6+V0TRPZ8YzKp2yQmDcGH\r\n69IbuKJ2BG1Qx8znZAvghKQ6P9Im+M4c7j9iMG72VKhDgmEwFB1acOw0lpu1XE8R1wmwG\r\nZRl/xzri3LOW2Gpc77xu6fs3NIkzQw3v1ifYhX3OrVsCIRBbDjPQI3yYjkhGx24s\r\nVhhZ5S/TkGk3Kw59bDC6KGqAuQAwX9req2l1NiuNaPU9rE7tf6Bk\r\n-----END RSA PRIVATE KEY-----"
    }
}

The ConsumerKey, ConsumerSecret, ConsumerName and PublicKeyPEM must be manually inserted into the "Application Links" section under JIRA Admin Settings by a JIRA admin on the target JIRA installation. Once that is complete, users can OAuth on the target JIRA installation.

Make a request for JIRA Auth

curl -X POST localhost:4050/admin/requestAuthSession --data-binary '{
    "RealmID": "jirarealm",
    "UserID": "@example:localhost",
    "Config": {
        "RedirectURL": "https://optional-url.com/to/redirect/to/after/auth"
    }
}'

Returns:

{
    "URL":"https://jira.somewhere.com/plugins/servlet/oauth/authorize?oauth_token=7yeuierbgweguiegrTbOT"
}

Follow this link and grant access for NEB to act on your behalf.

Create a JIRA bot

curl -X POST localhost:4050/admin/configureService --data-binary '{
    "Type": "jira",
    "Id": "jid",
    "UserID": "@goneb:localhost",
    "Config": {
        "ClientUserID": "@example:localhost",
        "Rooms": {
            "!EmwxeXCVubhskuWvaw:localhost": {
                "Realms": {
                    "jira_realm_id": {
                        "Projects": {
                            "BOTS": {
                                "Expand": true,
                                "Track": true
                            }
                        }
                    }
                }
            }
        }
    }
}'

Starting a Giphy Service

Create a Giphy bot

curl -X POST localhost:4050/admin/configureService --data-binary '{
    "Type": "giphy",
    "Id": "giphyid",
    "UserID": "@goneb:localhost",
    "Config": {
        "APIKey": "YOUR_API_KEY"
    }
}'

Developing on go-neb.

There's a bunch more tools this project uses when developing in order to do things like linting. Some of them are bundled with go (fmt and vet) but some are not. You should install the ones which are not:

go get github.com/golang/lint/golint
go get github.com/fzipp/gocyclo

You can then install the pre-commit hook:

./hooks/install.sh

Viewing the API docs.

# Start a documentation server listening on :6060
GOPATH=$GOPATH:$(pwd) godoc -v -http=localhost:6060 &

# Open up the documentation for go-neb in a browser.
sensible-browser http://localhost/pkg/github.com/matrix-org/go-neb

18 KiB Raw Blame History

Go-NEB

Table of Contents

Quick Start

Features

Github

JIRA

Giphy

Installing

Running

Configuring Clients

Configuring Services

Echo Service

Github Service

JIRA Service

Giphy Service

Developing

Building go-neb

Running go-neb

Architecture

Starting a Github Service

Register a Github realm

Make a request for Github Auth

Create a github bot

Starting a JIRA Service

Register a JIRA realm

Make a request for JIRA Auth

Create a JIRA bot

Starting a Giphy Service

Create a Giphy bot

Developing on go-neb.

Viewing the API docs.

18 KiB

Raw Blame History