A lot of the code in common between this and the EmbedlyScraper should
probably be generalized out to a base class soon, but let's make sure
this works first.
I hate almost everything about this, but it seems to work fine overall.
Pagination is still restricted to users themselves currently, so you can
only page back through history and switch to topics or comments only on
yourself. This won't stay like this for much longer though.
Required telling prospector to ignore some abstract methods inside
wrapt's ObjectProxy - they're not truly abstract, but will raise
NotImplementedError if they're called, so I guess pylint detects those
as abstract.
The monitoring server needs Redis, but not the separate server that's
used for the breached-passwords bloom filter in dev/prod. This splits
that server out to its own state, so that it doesn't need to be set up
on the monitoring server.
Some of these states were built entirely around a single-server approach
(Prometheus + monitoring being on the same server as the site), and the
files have needed modifications to work with a separate monitoring
server.
This updates the states so that it should all happen as expected in all
types of environments.
I'm seeing some errors come through now when requests for unhandled urls
(things like /ads.txt) are made, usually just by various bots/scanners.
The metrics tween is crashing on these, so this should fix it.
When a form status message is displayed (often an error), it could cause
the button to be shrunk, making it look strange (sometimes the text
would even become larger than the button background). This prevents it
from being able to shrink and will cause the message to wrap instead.
Previously I was using Salt to install the Sentry SDK (previously known
as "raven") only on the production server, but that's not really
necessary. This will just install it everywhere, and then we'll only
actually integrate it in production.
Now that all links in text have underlines by default, I think this
looks pretty strange for ~group and @user links, which are quite common
and unnecessary to have underlined all the time. This modifies the
markdown parser to add link-user and link-group classes to these links,
which allows them to be styled differently.
In addition, some of the markdown tests needed to be changed to use
BeautifulSoup instead of simple string-matching, since it's not as
simple to recognize links any more (and the order of attrs might
change).
I think this is a good idea for a few reasons, including accessibility
(people that have difficulty distinguishing the link color will still be
able to recognize links).
This is a bit flimsy, but when I started looking at applying the
existing transformations to old posts, I found the Paradox forums as an
example of links that became broken after they were processed (because
"fixing" their links ends up breaking them).
This will give a way to exempt any other domains or urls that end up
being a problem, though over the long term it would probably be better
to make this database-based instead of code-based.
webargs 5.0.0 makes a behavior change to its @use_args / @use_kwargs
decorators that makes it so that optional/missing arguments are no
longer filled: https://github.com/marshmallow-code/webargs/issues/342
This breaks how I'm currently doing some views with missing arguments
(such as views.topic.get_group_topics), so to be able to upgrade to 5.0
we will need to either update the views or the schemas.
When a topic tag was using the hierarchy (for something like
"science.biology"), it would set an invalid class on the tag label,
because it would include the period(s), which can't be used in CSS
classes. This fixes it so that periods will be replaced with dashes.
This is (extreme) overkill at this point, but I'm thinking ahead to some
of the transformations that I want to be able to support, and this
refactor should make it much more feasible to have a good process.
The main changes here are using a class-based system where each
transformation can also define a method for checking whether it should
apply to any given url (probably often based on domain), as well as a
new process that will repeatedly apply all transformations until the url
passes through all of them unchanged.
This script can be scheduled as a cronjob, and will dump the database,
compress and GPG-encrypt it, and upload to an FTP. Afterwards, it will
also delete any backups older than the specified retention periods, both
locally as well as on the FTP (with individual retention periods).
Since the check to see if a password has been present in a data breach
is using a Bloom filter, there's a small chance of false positives (I
believe it's 0.1% currently). This is confusing when it happens, so this
just clarifies that it's possible but they'll have to pick a new
password anyway.
Previously, dates were always displayed in the relative, "ago" style,
but these become pretty unwieldy for longer time spans, especially since
the ago library I'm using jumps directly from days to years, so it will
show ones like "200 days ago" that are hard to place.
This adds an "adaptive" date display method and uses it almost
everywhere instead. These will use the "ago" style for shorter periods,
and then switch to showing an absolute date for dates longer ago than
that. The threshold for the switch is currently set to 7 days.
This should be more correct overall, and in the (rare) cases where the
destination changes (due to a topic being moved to a different group or
something similar), the site is able to handle it with a 302 redirect
after the initial one from the shortener.
I did a bad job of testing how this would work, and didn't account
properly for people being considered not-logged-in while they're on
tild.es. This should fix the issues, but it re-adds the minor "data
leak" of people without an invite being able to determine some data,
such as existing users and groups. Since the site is going to be
publicly visible in the near future, I don't think this is a significant
concern.
Links aren't displayed/used anywhere yet, but this should be the basic
setup needed for a simple link-shortener on the tild.es domain.
Currently, it will support two uses:
* https://tild.es/asdf - redirect to topic with id "asdf"
* https://tild.es/~asdf - redirect to group "~asdf"
Deimos
deing
Chad Birch