Kegan Dougal
38ead2c174
Make ServiceDB an interface called `Storer`
Implement a nop `Storer` called `NopStorage` which tests can make use of.
This was awful to write.
8 years ago
Kegsay
2a980eee21
Merge pull request #103 from matrix-org/kegan/rss-escape-entities
RSS: HTML decode the item/description fields
8 years ago
Kegan Dougal
6fd0f20c81
Review comments
8 years ago
Kegan Dougal
fcd3befb09
HTML decode the RSS title/description fields
8 years ago
Kegan Dougal
07e93c5ba2
Add failing TestHTMLEntities test
8 years ago
Kegan Dougal
ba6e5e8862
Rotate logs on a daily basis: Fixes #97
8 years ago
Kegsay
94e2c72451
Merge pull request #102 from matrix-org/kegan/tests
Move test util functions to dedicated file to clean up test files
8 years ago
Kegan Dougal
f753e0b669
Move test util functions to dedicated file to clean up test files
Also use verbose mode on tests so you can see which test is running when things
print to stdout.
8 years ago
Kegsay
c341adaaf2
Merge pull request #101 from matrix-org/kegan/tests-prep
Make it possible to do integration tests
8 years ago
Kegan Dougal
352d741558
Review comments
8 years ago
Kegan Dougal
5c65d4cf95
Add a mock HTTP client for Matrix clients to use. Add example test.
This will need rejigging at some point to make the test easier to set up.
8 years ago
Kegan Dougal
dc2fadd450
Factor out startup process to `setup()` and add top-level test
- Pass in an `envVars` struct rather than grabbing directly from `os`.
- Accept a `ServeMux` rather than always using the default mux.
- Add a `_test.go` file to instrument Go-NEB using `TestMain` to call `setup()`
8 years ago
Kegsay
2b0e244d3f
Merge pull request #100 from matrix-org/kegan/service-isolation
Prevent panicking code from taking down the entire process
8 years ago
Kegan Dougal
236c97d279
Tests and comments
8 years ago
Kegan Dougal
92ac46e925
Factor out a Protect() function for incoming HTTP requests
8 years ago
Kegan Dougal
7ab9794623
Protect panicking poll loops from taking down the entire process
8 years ago
Kegan Dougal
ce414ea586
Prevent panicking code triggered by matrix events from taking down the entire process
Specifically, this protects panicking `!commands` and expansions.
8 years ago
Kegan Dougal
3efb632b31
Log panicking HTTP requests
Whilst Go does this by default (does not `exit`), it logs to the std logger
which isn't written to the log file. Handle all panicking `OnIncomingRequests`.
8 years ago
Kegsay
53d46a9e5d
Mention the config file
8 years ago
Kegsay
9d059d2e55
Merge pull request #98 from matrix-org/kegan/config-yaml
Support config.yaml in addition to HTTP endpoints
8 years ago
Kegan Dougal
9d9934e923
Tidy up how we load the database
8 years ago
Kegan Dougal
f40803ba6b
Flesh out config file to explain gotchas
8 years ago
Kegan Dougal
79c3145cca
Docs for Check() functions
8 years ago
Kegan Dougal
e4cacb1cd5
Use correct key names for service configs
8 years ago
Kegan Dougal
08d4db20bf
Fix panic if github is down
8 years ago
Kegan Dougal
0fce8aea09
Glue everything together
8 years ago
Kegan Dougal
295c9bbb4b
Add 'api' package and move HTTP API requests there
This will make docs easier in addition to allowing the 'database' package
not circular-dep.
8 years ago
Kegan Dougal
56e92a9632
Convert YAML to NEB types via roundabout methods
8 years ago
Kegan Dougal
a7de9a7fdd
Add YAML tags to things so we can unmarshal them. Cannot partially unmarshal json.RawMessage-like though...
8 years ago
Kegan Dougal
bbf89c13bc
Merge branch 'master' into kegan/config-yaml
8 years ago
Kegan Dougal
36562c6f2c
Add YAML parser
8 years ago
Kegan Dougal
f4033d1745
Add stub config file loader
8 years ago
Kegan Dougal
7beaf57ff7
Merge branch 'master' into kegan/config-yaml
8 years ago
Kegan Dougal
e35ea2f27c
Use the NextPollTimestampSecs or else we'll resend on /configureService due to JSON key name change
8 years ago
Kegsay
1cb7a3b565
Merge pull request #94 from matrix-org/kegan/rss-notify-on-failures
Add a flag `IsFailing` to let clients know if their RSS feeds are down
8 years ago
Kegan Dougal
bc1e6283d8
Make some fields 'public'
8 years ago
Kegan Dougal
fae997fdb7
Add a flag `IsFailing` to let clients know if their RSS feeds are down
8 years ago
Kegan Dougal
3c7cf89995
Add a sample config file
8 years ago
Kegan Dougal
299164d525
Fix RSS feeds which do not return GUIDs and do not have published dates
We previously relied on the published date and GUIDs to determine which items
were new. We now only rely on the GUID and not the published date as a lot of
RSS feeds don't have published dates. A lot of feeds don't have GUIDs either,
so we now fallback to the HTTP `Link` field, or worst-case, the item `Title`.
8 years ago
Kegsay
1f47c5a202
Merge pull request #92 from matrix-org/kegan/rss-user-agent
Set "Go-NEB" as the User-Agent on RSS feed polling
8 years ago
Kegan Dougal
5b5de2dc4b
Set "Go-NEB" as the User-Agent on RSS feed polling
Some services, notably Reddit, basically force you to set a custom UA in order
to use their RSS feeds. All the common default non-browser UAs are HEAVILY
rate-limited such that you can't realistically use them.
8 years ago
Kegan Dougal
5c8cc4a81c
strconv to convert int to string
8 years ago
Kegan Dougal
433331ca20
Metrics: Add auth session counter
8 years ago
Kegsay
3f31f63b19
Merge pull request #91 from matrix-org/kegan/neb-metrics
Add metrics to NEB
8 years ago
Kegan Dougal
f493a1933e
s/sendMetric/incrementMetrics/
8 years ago
Kegan Dougal
2e5c8e4d79
Only send the domain part of the RSS feed as that's more useful when doing aggregate metrics
8 years ago
Kegan Dougal
b4b0a661c4
Add more metrics
8 years ago
Kegan Dougal
cdb49fa60b
Add command type as a label
8 years ago
Kegan Dougal
32e8a5b6cc
Docs
8 years ago
Kegan Dougal
19eeb96610
Use enums instead
8 years ago