go-neb

Commit Graph

Author	SHA1	Message	Date
Kegan Dougal	5b5de2dc4b	Set "Go-NEB" as the User-Agent on RSS feed polling Some services, notably Reddit, basically force you to set a custom UA in order to use their RSS feeds. All the common default non-browser UAs are HEAVILY rate-limited such that you can't realistically use them.	8 years ago
Kegan Dougal	5c8cc4a81c	strconv to convert int to string	8 years ago
Kegan Dougal	433331ca20	Metrics: Add auth session counter	8 years ago
Kegsay	3f31f63b19	Merge pull request #91 from matrix-org/kegan/neb-metrics Add metrics to NEB	8 years ago
Kegan Dougal	f493a1933e	s/sendMetric/incrementMetrics/	8 years ago
Kegan Dougal	2e5c8e4d79	Only send the domain part of the RSS feed as that's more useful when doing aggregate metrics	8 years ago
Kegan Dougal	b4b0a661c4	Add more metrics	8 years ago
Kegan Dougal	cdb49fa60b	Add command type as a label	8 years ago
Kegan Dougal	32e8a5b6cc	Docs	8 years ago
Kegan Dougal	19eeb96610	Use enums instead	8 years ago
Kegan Dougal	219da01e08	Add extremely noddy counter metrics for !commands Missing the command type currently. Also, not sure how "nice" this is: we could probably blob the 3 functions into 1 and use an enum but, meh?	8 years ago
Kegan Dougal	729b80c6bc	Set max open conns to 1 on sqlite3	8 years ago
Kegsay	e875b9c251	Merge pull request #87 from matrix-org/kegan/metrics Instrument all HTTP handlers	8 years ago
Kegan Dougal	c9de6eaf06	Instrument all HTTP handlers	8 years ago
Kegan Dougal	d169b5560f	Add prometheus dep for metrics	8 years ago
Kegan Dougal	248822a3d6	Fix #86 - set a 20s min threshold before re-polling	8 years ago
Kegan Dougal	ef6fc0d3bf	Remove useless field	8 years ago
Kegsay	7bcffa4391	Merge pull request #85 from matrix-org/kegan/database-is-locked Edit README to put `_busy_timeout` to fix database is locked errors (#63)	8 years ago
Kegsay	d9da3caac4	Merge pull request #84 from matrix-org/kegan/rss-guids Maintain a list of GUIDs per feed and use them to de-dupe	8 years ago
Kegan Dougal	95521a08f2	Edit README to put `_busy_timeout` to fix database is locked errors (#63 )	8 years ago
Kegan Dougal	8316ac21ab	Maintain a list of GUIDs per feed and use them to de-dupe Some RSS feeds will edit the published time of an item AFTER putting it out, which resulted in RSS Bot sending the same article twice. We now remember the "GUID" field for each item and de-dupe based on that. Normal timestamp algorithm still applies.	8 years ago
Kegan Dougal	77b729d4ba	Remove spammy logs	8 years ago
Kegan Dougal	5f015e26ee	Join rooms when registering rss service	8 years ago
Kegan Dougal	b903860af9	Bump min to 5min to help not spam feeds with no cache control	8 years ago
Kegan Dougal	1640728857	Add lrucache	8 years ago
Kegsay	2f0a2c7f10	Merge pull request #83 from matrix-org/kegan/cache-rss-feeds Add a caching HTTP client	8 years ago
Kegan Dougal	e088e11821	Swap to an LRU cache set to 20MB	8 years ago
Kegan Dougal	e61356431a	Merge branch 'master' into kegan/cache-rss-feeds	8 years ago
Kegan Dougal	7587dbc9c4	Add lrucache	8 years ago
Kegan Dougal	b0eb284347	Add a caching HTTP client	8 years ago
Kegan Dougal	223ccac851	Add HTTP caching library	8 years ago
Kegsay	f63e5ce7c1	Merge pull request #82 from matrix-org/kegan/per-service-poll Move OnPoll into the Service method set	8 years ago
Kegan Dougal	fd54eb1a55	Move OnPoll into the Service method set This feels a lot better because now `OnPoll` works in a similar way to `OnReceiveWebhook` (called on a `Service`) rather than have this strange global-per-service-type struct.	8 years ago
Kegan Dougal	7518ed1995	Fix NPE if oldService is nil	8 years ago
Kegsay	5fe9743cba	Merge pull request #81 from matrix-org/kegan/rssbot s/feedreader/rssbot/g	8 years ago
Kegan Dougal	cd39219113	s/feedreader/rssbot/g Because apparently "Feed Reader" is the name of a thing popular enough that we don't want to step on toes, so let's call it the name of a less popular thing.	8 years ago
Kegsay	ccb1c6c779	Merge pull request #80 from matrix-org/kegan/feedreader-struct Rejig how feedreaders stores rooms to send updates to	8 years ago
Kegan Dougal	185a7d5678	Rejig how feedreaders stores rooms to send updates to	8 years ago
Kegsay	7971b13368	Merge pull request #78 from matrix-org/kegan/feedreader Implement feedreader service	8 years ago
Kegan Dougal	8684b671e2	Bump sqlite3 dep to make it compile on OSX	8 years ago
Kegan Dougal	7b4448957b	Review comments	8 years ago
Kegan Dougal	98e7533469	Send feed items in reverse-order for chronological times Also add checks on Register() to make sure the feeds are valid	8 years ago
Kegan Dougal	d92a95ff4e	Linting	8 years ago
Kegan Dougal	0b09fcd86d	First working feed reader version	8 years ago
Kegan Dougal	2d6b3cd6e9	Flesh out the feedreader service Just need to send messages into rooms now for a first cut to be done. Notable improvements to make: - We currently do 1 goroutine per service. This could be bad if we have lots of these things running around. - We do not cache the response to RSS feeds. If we have 10 independent services on the same feed URL, we will hit the URL 10 times. This is similar to how we currently do 1 webhook/service, so it's plausible that in the future we will want to have some kind of generic caching layer. - We don't send messages to Matrix yet. We need a `Clients` instance but can't get at one. There's only ever one, so I wonder if we should global it like we do with `GetServiceDB()` for ease of use? - The polling interval is divorced from the actual feed repoll time. Ideally we would schedule the goroutine only when we need it, rather than checking frequently, determining we have nothing to do, and going back to sleep.	8 years ago
Kegsay	d8c1991cf6	Create LICENSE	8 years ago
Kegan Dougal	a5e515e118	Lint	8 years ago
Kegan Dougal	3e3f71ad92	s/rss/feedreader/ since it does atom feeds too. Add TODO	8 years ago
Kegan Dougal	825b3e74b3	Add github.com/mmcdole/gofeed - Make guggy use DefaultService	8 years ago
Kegsay	c6a15801eb	Merge pull request #76 from matrix-org/kegan/rss Implement polling for Services	8 years ago

1 2 3 4 5 ...

280 Commits (5b5de2dc4b26a2643329298cffed3a2b0ffe8be9) All Branches Search

280 Commits (5b5de2dc4b26a2643329298cffed3a2b0ffe8be9)

All Branches