Branch: kegan/all-the-docs

anoa/go1.11

anoadragon453-patch-1

dependabot/go_modules/github.com/prometheus/client_golang-1.11.1

dependabot/go_modules/golang.org/x/net-0.7.0

dependabot/go_modules/golang.org/x/sys-0.1.0

feature-slack-api

gh-pages

hs/check-pls

hs/discontinued

hs/remove-rss-url-metric

hs/search-usage

kegan/all-the-docs

kegan/auth

kegan/auth-realms

kegan/auth-session

kegan/auth-session-api-tweak

kegan/base64-paths

kegan/better-gh-cmd-regex

kegan/bot-options

kegan/cache-rss-feeds

kegan/case-insensitive-commands

kegan/config-yaml

kegan/configure-client-name

kegan/create-with-json

kegan/database-is-locked

kegan/db-interface

kegan/docs-services

kegan/dots-in-repos

kegan/expand-with-groups

kegan/fail-if-bad-user

kegan/feed-leak

kegan/feedreader

kegan/feedreader-struct

kegan/finer-gh-controls

kegan/fix-log-field-names

kegan/get-service

kegan/get-session

kegan/gh-explicit-unregister-webhooks

kegan/gh-help

kegan/gh-logout

kegan/gh-no-webhooks-means-no-service

kegan/gh-service-client-id-is-opt

kegan/giphy

kegan/giphy-translate-api

kegan/github

kegan/github-auth

kegan/github-cmd-expan-service

kegan/github-cmds

kegan/github-join-room-on-register

kegan/github-make-webhooks-on-register

kegan/github-mutex-on-register

kegan/github-possible-repos

kegan/github-remove-hooks-on-webhook-event

kegan/github-say-which-labels

kegan/github-service

kegan/github-starter-links

kegan/github-webhook

kegan/github-webhook-creation

kegan/gomatrix

kegan/guggy-tests

kegan/httpcache-update

kegan/ignore-events-on-join

kegan/jira-auth

kegan/jira-auth-sessions

kegan/jira-commands

kegan/jira-expansion

kegan/jira-realm

kegan/jira-redirect

kegan/jira-starter-links

kegan/jira-webhook-2

kegan/jira-webhooks

kegan/jsonresponse

kegan/logging

kegan/matrix-org-util

kegan/metrics

kegan/min-guid-length

kegan/move-handlers

kegan/neb-metrics

kegan/per-service-poll

kegan/persist-next-batch

kegan/readme-to-docs

kegan/realm-docs

kegan/redirect

kegan/remove-docs-from-readme

kegan/remove-plugin

kegan/rm-3pauth

kegan/rss

kegan/rss-dont-replace-if-no-guids

kegan/rss-escape-entities

kegan/rss-guids

kegan/rss-limits

kegan/rss-no-cache

kegan/rss-notify-on-failures

kegan/rss-user-agent

kegan/rssbot

kegan/service-docs

kegan/service-isolation

kegan/shut-up-golint

kegan/smart-quotes

kegan/split-gh-service

kegan/sync-on-startup

kegan/tests

kegan/tests-prep

kegan/testutils

kegan/text-logging

kegan/travis

kegan/webhook-endpoint-fn

kegan/webhook-path-as-id

kegan/webhooks

luke/guggy

master

michaelkaye/log_alertmanager_errors

michaelkaye/respect_rate_limits

michaelkaye/upgrade-go

rob/docker

rxl881/google

rxl881/riotbot

t3chguy/github_regex_tweaks

t3chguy/google_safe_search

tadzik/fix-ci

0.1.0

0.1.1

0.1.2

Branches Tags

${ item.name }

Create tag ${ searchTerm }

Create branch ${ searchTerm }

from 'kegan/all-the-docs'

${ noResults }

Commit Graph

332 Commits (kegan/all-the-docs)

 

All Branches   

Search

Author	SHA1	Message	Date
Kegan Dougal	299164d525	Fix RSS feeds which do not return GUIDs and do not have published dates We previously relied on the published date and GUIDs to determine which items were new. We now only rely on the GUID and not the published date as a lot of RSS feeds don't have published dates. A lot of feeds don't have GUIDs either, so we now fallback to the HTTP `Link` field, or worst-case, the item `Title`.	8 years ago
Kegsay	1f47c5a202	Merge pull request #92 from matrix-org/kegan/rss-user-agent Set "Go-NEB" as the User-Agent on RSS feed polling	8 years ago
Kegan Dougal	5b5de2dc4b	Set "Go-NEB" as the User-Agent on RSS feed polling Some services, notably Reddit, basically force you to set a custom UA in order to use their RSS feeds. All the common default non-browser UAs are HEAVILY rate-limited such that you can't realistically use them.	8 years ago
Kegan Dougal	5c8cc4a81c	strconv to convert int to string	8 years ago
Kegan Dougal	433331ca20	Metrics: Add auth session counter	8 years ago
Kegsay	3f31f63b19	Merge pull request #91 from matrix-org/kegan/neb-metrics Add metrics to NEB	8 years ago
Kegan Dougal	f493a1933e	s/sendMetric/incrementMetrics/	8 years ago
Kegan Dougal	2e5c8e4d79	Only send the domain part of the RSS feed as that's more useful when doing aggregate metrics	8 years ago
Kegan Dougal	b4b0a661c4	Add more metrics	8 years ago
Kegan Dougal	cdb49fa60b	Add command type as a label	8 years ago
Kegan Dougal	32e8a5b6cc	Docs	8 years ago
Kegan Dougal	19eeb96610	Use enums instead	8 years ago
Kegan Dougal	219da01e08	Add extremely noddy counter metrics for !commands Missing the command type currently. Also, not sure how "nice" this is: we could probably blob the 3 functions into 1 and use an enum but, meh?	8 years ago
Kegan Dougal	729b80c6bc	Set max open conns to 1 on sqlite3	8 years ago
Kegsay	e875b9c251	Merge pull request #87 from matrix-org/kegan/metrics Instrument all HTTP handlers	8 years ago
Kegan Dougal	c9de6eaf06	Instrument all HTTP handlers	8 years ago
Kegan Dougal	d169b5560f	Add prometheus dep for metrics	8 years ago
Kegan Dougal	248822a3d6	Fix #86 - set a 20s min threshold before re-polling	8 years ago
Kegan Dougal	ef6fc0d3bf	Remove useless field	8 years ago
Kegsay	7bcffa4391	Merge pull request #85 from matrix-org/kegan/database-is-locked Edit README to put `_busy_timeout` to fix database is locked errors (#63)	8 years ago
Kegsay	d9da3caac4	Merge pull request #84 from matrix-org/kegan/rss-guids Maintain a list of GUIDs per feed and use them to de-dupe	8 years ago
Kegan Dougal	95521a08f2	Edit README to put `_busy_timeout` to fix database is locked errors (#63)	8 years ago
Kegan Dougal	8316ac21ab	Maintain a list of GUIDs per feed and use them to de-dupe Some RSS feeds will edit the published time of an item AFTER putting it out, which resulted in RSS Bot sending the same article twice. We now remember the "GUID" field for each item and de-dupe based on that. Normal timestamp algorithm still applies.	8 years ago
Kegan Dougal	77b729d4ba	Remove spammy logs	8 years ago
Kegan Dougal	5f015e26ee	Join rooms when registering rss service	8 years ago
Kegan Dougal	b903860af9	Bump min to 5min to help not spam feeds with no cache control	8 years ago
Kegan Dougal	1640728857	Add lrucache	8 years ago
Kegsay	2f0a2c7f10	Merge pull request #83 from matrix-org/kegan/cache-rss-feeds Add a caching HTTP client	8 years ago
Kegan Dougal	e088e11821	Swap to an LRU cache set to 20MB	8 years ago
Kegan Dougal	e61356431a	Merge branch 'master' into kegan/cache-rss-feeds	8 years ago
Kegan Dougal	7587dbc9c4	Add lrucache	8 years ago
Kegan Dougal	b0eb284347	Add a caching HTTP client	8 years ago
Kegan Dougal	223ccac851	Add HTTP caching library	8 years ago
Kegsay	f63e5ce7c1	Merge pull request #82 from matrix-org/kegan/per-service-poll Move OnPoll into the Service method set	8 years ago
Kegan Dougal	fd54eb1a55	Move OnPoll into the Service method set This feels a lot better because now `OnPoll` works in a similar way to `OnReceiveWebhook` (called on a `Service`) rather than have this strange global-per-service-type struct.	8 years ago
Kegan Dougal	7518ed1995	Fix NPE if oldService is nil	8 years ago
Kegsay	5fe9743cba	Merge pull request #81 from matrix-org/kegan/rssbot s/feedreader/rssbot/g	8 years ago
Kegan Dougal	cd39219113	s/feedreader/rssbot/g Because apparently "Feed Reader" is the name of a thing popular enough that we don't want to step on toes, so let's call it the name of a less popular thing.	8 years ago
Kegsay	ccb1c6c779	Merge pull request #80 from matrix-org/kegan/feedreader-struct Rejig how feedreaders stores rooms to send updates to	8 years ago
Kegan Dougal	185a7d5678	Rejig how feedreaders stores rooms to send updates to	8 years ago
Kegsay	7971b13368	Merge pull request #78 from matrix-org/kegan/feedreader Implement feedreader service	8 years ago
Kegan Dougal	8684b671e2	Bump sqlite3 dep to make it compile on OSX	8 years ago
Kegan Dougal	7b4448957b	Review comments	8 years ago
Kegan Dougal	98e7533469	Send feed items in reverse-order for chronological times Also add checks on Register() to make sure the feeds are valid	8 years ago
Kegan Dougal	d92a95ff4e	Linting	8 years ago
Kegan Dougal	0b09fcd86d	First working feed reader version	8 years ago
Kegan Dougal	2d6b3cd6e9	Flesh out the feedreader service Just need to send messages into rooms now for a first cut to be done. Notable improvements to make: - We currently do 1 goroutine per service. This could be bad if we have lots of these things running around. - We do not cache the response to RSS feeds. If we have 10 independent services on the same feed URL, we will hit the URL 10 times. This is similar to how we currently do 1 webhook/service, so it's plausible that in the future we will want to have some kind of generic caching layer. - We don't send messages to Matrix yet. We need a `Clients` instance but can't get at one. There's only ever one, so I wonder if we should global it like we do with `GetServiceDB()` for ease of use? - The polling interval is divorced from the actual feed repoll time. Ideally we would schedule the goroutine only when we need it, rather than checking frequently, determining we have nothing to do, and going back to sleep.	8 years ago
Kegsay	d8c1991cf6	Create LICENSE	8 years ago
Kegan Dougal	a5e515e118	Lint	8 years ago
Kegan Dougal	3e3f71ad92	s/rss/feedreader/ since it does atom feeds too. Add TODO	8 years ago